ABSTRACT 


Title  of  Document:  SOLVING  TWO-LEVEL  OPTIMIZATION 

PROBLEMS  WITH  APPLICATIONS  TO 
ROBUST  DESIGN  AND  ENERGY  MARKETS 

Sauleh  Ahmad  Siddiqui 
Doctor  of  Philosophy,  2011 

Directed  By:  Steven  A.  Gabriel,  Associate  Professor 

Department  of  Civil  and  Environmental 
Engineering 

Shapour  Azarm,  Professor 
Department  of  Mechanical  Engineering 


This  dissertation  provides  efficient  techniques  to  solve  two-level  optimization 
problems.  Three  specific  types  of  problems  are  considered.  The  first  problem  is 
robust  optimization,  which  has  direct  applications  to  engineering  design. 
Traditionally  robust  optimization  problems  have  been  solved  using  an  inner-outer 
structure,  which  can  be  computationally  expensive.  This  dissertation  provides  a 
method  to  decompose  and  solve  this  two-level  structure  using  a  modified  Benders 
decomposition.  This  gradient-based  technique  is  applicable  to  robust  optimization 
problems  with  quasiconvex  constraints  and  provides  approximate  solutions  to 
problems  with  nonlinear  constraints.  The  second  types  of  two-level  problems 
considered  are  mathematical  and  equilibrium  programs  with  equilibrium  constraints. 
Their  two-level  structure  is  simplified  using  Schur"s  decomposition  and  reformulation 


Report  Documentation  Page 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

2011 


2.  REPORT  TYPE 


3.  DATES  COVERED 

00-00-2011  to  00-00-2011 


4.  TITLE  AND  SUBTITLE 

Solving  Two-Level  Optimization  Problems  With  Applications  To  Robust 
Design  And  Energy  Markets 

6.  AUTHOR(S) 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

University  of  Maryland, College  Park,MD,  20782 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


5a.  CONTRACT  NUMBER 


5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

5d.  PROIECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 


13.  SUPPLEMENTARY  NOTES 


14.  ABSTRACT 


15.  SUBIECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 

unclassified 


b.  ABSTRACT 

unclassified 


c.  THIS  PAGE 

unclassified 


17.  LIMITATION  OF 

18.  NUMBER 

ABSTRACT 

OF  PAGES 

Same  as 

222 

Report  (SAR) 

19a.  NAME  OF 
RESPONSIBLE  PERSON 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


schemes  for  absolute  value  functions.  The  resulting  fonnulations  are  applicable  to 
game  theory  problems  in  operations  research  and  economics.  The  third  type  of  two- 
level  problem  studied  is  discretely-constrained  mixed  linear  complementarity 
problems.  These  are  first  formulated  into  a  two-level  mathematical  program  with 
equilibrium  constraints  and  then  solved  using  the  aforementioned  technique  for 
mathematical  and  equilibrium  programs  with  equilibrium  constraints.  The  techniques 
for  all  three  problems  help  simplify  the  two-level  structure  into  one  level,  which  helps 
gain  numerical  and  application  insights.  The  computational  effort  for  solving  these 
problems  is  greatly  reduced  using  the  techniques  in  this  dissertation.  Finally,  a  host  of 
numerical  examples  are  presented  to  verify  the  approaches.  Diverse  applications  to 
economics,  operations  research,  and  engineering  design  motivate  the  relevance  of  the 
novel  methods  developed  in  this  dissertation. 


SOLVING  TWO-LEVEL  OPTIMIZATION  PROBLEMS  WITH  APPLICATIONS 
TO  ROBUST  DESIGN  AND  ENERGY  MARKETS 


By 


Sauleh  Ahmad  Siddiqui 


Dissertation  submitted  to  the  Faculty  of  the  Graduate  School  of  the 
University  of  Maryland,  College  Park,  in  partial  fulfillment 
of  the  requirements  for  the  degree  of 
Doctor  of  Philosophy 
2011 


Advisory  Committee: 

Associate  Professor  Steven  A.  Gabriel,  Co-Advisor/Chair 

Professor  Shapour  Azarm,  Co-Advisor 

Associate  Professor  Radu  V.  Balan 

Professor  Dianne  P.  O'Leary 

Professor  Lars  J.  Olson,  Dean"s  Representative 


©  Copyright  by 
Sauleh  Ahmad  Siddiqui 
2011 


Acknowledgements 


First  and  foremost,  I  would  like  to  thank  my  advisers  Dr.  Steven  A.  Gabriel  and  Dr. 
Shapour  Azarm  for  all  their  help  and  support.  Their  perpetual  encouragement, 
abundant  patience,  and  careful  guidance  made  this  work  possible.  My  time  at  this 
university  was  made  memorable  because  of  them,  and  I  am  proud  to  look  back  and 
realize  how  much  I  have  learned  from  them. 

I  would  also  like  to  thank  my  committee  members  for  agreeing  to  advise  me 
on  this  work:  Dr.  Radu  V.  Balan,  for  overlooking  the  mathematical  aspects  in  both 
my  preliminary  oral  exam  and  dissertation;  Dr.  Dianne  P.  O'Leary,  whose  survival 
manual  for  graduate  study  in  the  computer  and  mathematical  sciences  provided 
valuable  advice;  and  Dr.  Lars  J.  Olson  for  graciously  agreeing  to  serve  as  the  Dean"s 
representative.  I  am  grateful  that  they  have  taken  time  out  from  their  busy  schedules 
to  provide  insight. 

I  also  want  to  acknowledge  funding  received  from  the  Office  of  Naval 
Research  and  the  Norwegian  Research  Council.  The  work  presented  in  this 
dissertation  was  supported  in  part  by  the  Office  of  Naval  Research  Contract 
N000140810384.  The  LinkS  project  funded  by  Norwegian  Research  Council  via  the 
Norwegian  University  of  Science  and  Technology  and  SINTEF  (UMD  Award 
number  013768-001)  also  supported  part  of  the  work  in  this  dissertation.  Such 
support  does  not  constitute  an  endorsement  by  the  funding  agency  of  the  opinions 
expressed  in  this  dissertation. 


n 


Table  of  Contents 


ACKNOWLEDGEMENTS . ii 

TABLE  OF  CONTENTS . iii 

LIST  OF  TABLES . vi 

LIST  OF  FIGURES . vii 

NOMENCLATURE  AND  ABBREVIATIONS . viii 

CHAPTER  1:  INTRODUCTION . 1 

1.1.  MOTIVATION  AND  OBJECTIVE . 1 

1.2.  RESEARCH  COMPONENTS . 3 

1.2.1.  Solving  Robust  Optimization  Problems . 3 

1.2.2.  Solving  Mathematical  Programs  and  Equilibrium  Problems  with 

Equilibrium  Constraints . 4 

1.2.3.  Solving  Discretely-Constrained  Mixed-Integer  Linear  Complementarity 

Problems . 4 

1.3.  ORGANIZATION  OF  DISSERTATION . 5 

CHAPTER  2:  DEFINITIONS  AND  LITERATURE  REVIEW . 7 

2.1.  INTRODUCTION . 7 

2.2.  DEFINITIONS  AND  TERMINOLOGIES . 7 

2.2.1.  Robust  Optimization . 9 

2.2.2.  Mathematical  and  Equilibrium  Programs  with  Equilibrium  Constraints  .  12 

2.2.3  Discretely-Constrained  Mixed  Linear  Complementarity  Problems . 16 

2.3.  OVERVIEW  OF  PREVIOUS  WORK . 19 

2.3.1  Robust  Optimization . 19 

2.3.3  Mathematical  and  Equilibrium  Programs  with  Equilibrium  Constraints  ..  23 

2.3.4.  Discretely-Constrained  Mixed  Linear  Complementarity  Problems . 24 

2.4.  PRELIMINARIES . 27 

2.4.1.  Benders  Decomposition . 27 

2.4.2.  Disjunctive  Constraints . 30 

2.4.3.  Approximating  Nonlinear  Functions  using  SOS  Type  1  and  Type  2 

Variables . 31 

CHAPTER  3:  SOLVING  ROBUST  OPTIMIZATION  PROBLEMS  USING  A 
MODIFIED  BENDERS  METHOD . 36 

3.1.  INTRODUCTION . 36 

3.2.  INTERVAL  UNCERTAINTY . 37 

3.3  MODIFIED  BENDERS  DECOMPOSITION . 54 

3.3.1.  Formulation  of  Approach:  Solving  Robust  Linear  Programs . 54 

3.3.2.  Formulation  of  Approach:  Solving  Robust  Optimization  Problems  with 

Quasiconvex  Constraints . 59 

3.3.3.  Formulation  of  Approach:  Solving  Robust  Optimization  Problems  with 

Nonlinear  Constraints . 63 

3.4.  NUMERICAL  RESULTS . 69 

3.4.1.  Numerical  Example  (Example  1)  to  Show  Methodology  Step-by-Step...  70 

3.4.2.  Numerical  Results . 74 

iii 


3.5.  ENGINEERING  DESIGN  AND  OTHER  APPLICATIONS . 76 

3.5.1.  Fleury"s  Weight  Minimization . 76 

3.5.2.  Design  of  a  Welded  Beam . 77 

3.5.3.  Heat  Exchanger  Design . 81 

3.5.4.  Building  Energy  Intensive  Infrastructure . 87 

3.6.  SUMMARY . 90 

CHAPTER  4:  SOLVING  MATHEMATICAL  PROGRAMS  AND 
EQUILIBRIUM  PROGRAMS  WITH  EQUILIBRIUM  CONSTRAINTS . 91 

4.1.  INTRODUCTION . 91 

4.2.  SOLVING  MATHEMATICAL  PROGRAMS  WITH  EQUILIBRIUM 

CONSTRAINTS . 92 

4.2.1.  Changing  the  Formulation  of  the  Lower-Level  Problem . 92 

4.2.2.  Approximating  The  Absolute  Value  Function  Using  Special  Ordered  Sets 

of  Type  1  Variables . 95 

4.2.3.  Approximating  Absolute  Value  Function  Using  a  Penalty  Method . 97 

4.2.4.  Algorithm  1  to  Solve  Mathematical  Programs  with  Equilibrium 

Constraints . 100 

4.2.5.  Numerical  Results . 101 

4.3.  SOLVING  EQUILIBRIUM  PROGRAMS  WITH  EQUILIBRIUM 

CONSTRAINTS . 112 

4.3.1.  Extending  Algorithm  4.1  to  Equilibrium  Programs  with  Equilibrium 

Constraints . 112 

4.3.2.  Algorithm  4.2  to  Solve  Equilibrium  Problems  with  Equilibrium 

Constraints  (Heuristic) . 114 

4.3.3.  Numerical  Results  for  Equilibrium  Programs  with  Equilibrium  Constraints 
. 115 

4.4.  THE  NORTH  AMERICAN  GAS  MODEL . 120 

4.4.1.  Introduction . 120 

4.4.2.  Shale  Gas  in  the  United  States . 122 

4.4.3.  Scenario  Results . 126 

4.5.  SUMMARY . 134 

CHAPTER  5:  SOLVING  DISCRETELY-CONSTRAINED  MIXED  LINEAR 
COMPLEMENTARITY  PROBLEMS . 135 

5.1.  INTRODUCTION . 135 

5.2.  DISCRETELY-CONSTRAINED  MIXED  LINEAR  COMPLEMENTARITY 

PROBLEMS . 137 

5.2.1  Epsilon-Integrality . 137 

5.2.2.  Sigma-Complementarity . 139 

5.2.3.  Complementarity,  Integrality  Trade-off. . 140 

5.2.4.  Formulation  to  Solve  Discretely-Constrained  Mixed  Linear 

Complementary  problems . 142 

5.3.  DISCRETELY-CONSTRAINED  NASH-COURNOT  GAMES . 144 

5.3.1.  Formulation  of  a  DC-Nash  game  by  Gabriel  et  al.  (2011b) . 145 

5.3.2.  First  Numerical  Example . 153 

5.3.3.  Results  for  First  Numerical  Example . 157 

5.3.4.  Numerical  Example  Relevant  to  Production  Systems . 160 


IV 


5.4.  DISCRETELY-CONSTRAINED  NETWORK  PROBLEMS . 163 

5.4.1.  First  Network  Example . 164 

5.4.2.  Second  Network  Example . 169 

5.5.  SUMMARY . 179 

CHAPTER  6:  CONCLUSIONS . 181 

6.1.  CONCLUDING  REMARKS . 181 

6.1.1.  Robust  Optimization . 181 

6.1.2.  MPECs  and  EPECs . 183 

6.1.3.  Discretely-Constrained  Mixed  Linear  Complementarity  Problems . 184 

6.2.  MAIN  CONTRIBUTIONS . 185 

6.3.  FUTURE  RESEARCH . 187 

6.3.1.  Multiobjective  Mixed-Integer  Robust  Optimization . 187 

6.3.2.  Solving  Nonlinear  MPECs  and  EPECs . 188 

6.3.3.  Solving  Large-Scale  Mixed-Integer  Complementary  Problems . 189 

APPENDICES . 190 

APPENDIX  A:  ROBUST  OPTIMIZATION  TEST  PROBLEMS . 190 

APPENDIX  B:  DISCUSSION  ON  FUNCTION  CALLS . 198 

BIBLIOGRAPHY . 201 


v 


List  of  Tables 


Table  2.1:  Definition  of  Terms  for  Robust  Optimization 
Table  2.2:  Definition  of  Terms  for  Benders  Decomposition 

Table  3.1:  Analysis  of  function  calls  for  one  iteration 
Table  3.2:  Solution  Steps  for  Modified  Benders  Approach 
Table  3.3:  Detailed  Solution  for  Simple  Problem 
Table  3.4:  Description  of  Test  Problems 

Table  3.5:  Results  for  Fleury"s  Weight  Minimization  Like  Problem 

Table  3.6:  Results  of  Welded  Beam  Example 

Table  3.7:  Design  Variables  and  Parameters  with  Uncertainty 

Table  3.8:  Results  for  Heat  Exchanger  Design 

Table  3.9:  Number  of  Iterations  and  CPU  Time  to  Solve  Problems 

Table  3.10:  Results  for  Increasing  Uncertainty  in  U 

Table  4.1:  Definition  of  terms  for  simple  example 

Table  4.2:  Different  Datasets  to  Compare  (4.13),  (4.15),  and  (4.16) 

Table  4.3:  Different  Cases  to  Compare  Solutions  to  (25) 

Table  4.4:  Results  for  Dataset  1 
Table  4.5:  Results  for  Dataset  2 
Table  4.6:  Results  for  Dataset  3 
Table  4.7:  Results  for  Dataset  1 
Table  4.8:  Results  for  Dataset  2 
Table  4.9:  Results  for  Dataset  3 

Table  4.10:  World  Gas  Model  Nodes:  Coverage  of  States  and  Shale  Basins 
Table  4.11:  Prices  in  $/MMBTU  in  2025 

Table  5.1:  Bimatrix  Nash-Cournot  Game,  Profits(c//A/?) 

Table  5.2:  Nash-Cournot  Game,  Profits(c///£/o),  (Only  Adjustments  a=9,  p2  =  3) 

Table  5.3:  Description  of  Formulation  Variations 

Table  5.4:  Summary  of  Results  (a  =  9,  b  =  1,  (3i=  (32  =  1,/A  =  1,  pz  =  3) 

Table  5.5:  Summary  of  Results  (a  =  9,  b  =  1,  (3i=  (I2  =  1  ,Pi  =  1 ,  Pz  =  3) 

Table  5.6:  Summary  of  Results  (Example  Relevant  to  Production  Systems) 

Table  5.7:  Summary  of  Results  (Example  Relevant  to  Production  Systems) 

Table  5.8:  Parameter  Values  Used  in  First  Network  Example 
Table  5.9:  Description  of  Formulation  Variations 
Table  5.10:  Solution  to  Power  Market  Example 
Table  5.11:  Dataset  Used  in  Second  Network  Example 

Table  5.12:  Description  of  Formulation  Variations  for  Second  Network  Example 
Table  5.13:  Results  for  Second  Network  Problem  (Integer  Variables) 

Table  5.14:  Results  for  Second  Network  Problem  (Other  Variables) 


vi 


List  of  Figures 


Figure  1.1:  Organization  of  Dissertation 


Figure  2.1: 
Figure  2.2: 
Figure  2.3: 
Figure  2.4: 
Figure  2.5: 
Figure  2.6: 
Figure  2.7: 


The  Structure  of  a  Two-Level  Problem 
Representation  of  a  Robust  Optimization  Problem 
Representation  of  an  MPEG 
Representation  of  an  EPEC 
Representation  of  a  DC-MLCP 

Approximating  a  Nonlinear  Function  Using  SOS  Type  1  Variable 
Approximation  of  Nonlinear  Functions  using  SOS  Type  2  Variables 


Figure 

Figure 

Figure 

Figure 

Figure 

Figure 


3.1:  Comparison  of  the  Feasible  Region  with  the  Robust  Feasible  Region 
3.2:  The  Robust  Benders  Cuts  to  Estimate  the  Maximum  Endpoint  of  au 
3.3:  Checking  Feasibility  by  Interval-Optimal  Points  for  Constraints 
3.4:  Adding  a  modified  (Robust)  Benders  Cut 
3.5:  Design  of  a  Welded  Beam  (Gunawan  &  Azarm,  2004) 

3.6:  Heat  Exchanger  Schematic 


Figure  4.1: 
Figure  4.2: 
Figure  4.3: 
Figure  4.4: 
Figure  4.5: 
Figure  4.6: 
Figure  4.7: 


Computational  Time  for  Solving  Problem 
A  Marginal  Cost  Structure  for  Shale  Gas  (Skagen,  2010) 
A  Marginal  Cost  Structure  for  Shale  Gas 
Overall  Production  in  2025  as  Predicted  by  the  Model 
Producer  Profit  in  2025  as  Predicted  by  the  Model 
Shale  Producers  in  2025  as  Predicted  by  the  Model 
Consumption  in  2025  as  Predicted  by  the  Model 


Figure  5.1: 
Figure  5.2: 
Figure  5.3: 
Figure  5.4: 
Figure  5.5: 


The  Tradeoff  Between  Complementary  and  Integrality 
Computational  Time  for  First  Numerical  Example 
Computational  Time  for  Example  Relevant  to  Production  Systems 
Diagram  of  First  Network  Example 
Representation  of  Second  Network  Example 


vii 


Nomenclature  and  Abbreviations 


BCM 

DC-MLCP 

DC-Nash 

EPEC 

/ 

g 

KKT 

LBF 

LP 

MCF 

MIP 

MILP 

MMBTU 

MPEC 

NCP 

91 

SOS 

x 

WGM 

Z 


Billion  Cubic  Meters 

Discretely-Constrained  Mixed  Linear  Complementary  Problems 
Discretely-Constrained  Nash  Game 
Equilibrium  Program  with  Equilibrium  Constraints 
Objective  Function  (Unless  stated  otherwise) 

Inequality  Constraint  Function  (Unless  stated  otherwise) 

Karush-Kuhn-Tucker  Conditions 

Pound  Force 

Linear  Program 

Million  Cubic  Feet 

Mixed  Integer  Program 

Mixed  Integer  Linear  Program 

One  million  British  Thermal  Units;  measurement  of  heat  energy 

Mathematical  Programs  with  Equilibrium  Constraints 

Nonlinear  Complementary  Problem 

The  real  numbers 

Special  Ordered  Set 

Decision  Variable 

World  Gas  Model 

The  Integers 


viii 


Chapter  1:  Introduction 


LL  Motivation  and  Objective 

Mathematical  modeling  of  problems  arising  in  engineering  and  economics  often 
requires  formulations  where  optimal  decisions  need  to  be  made  at  two  different 
levels.  These  levels  can  be  distinguished  by  time,  space,  decision  choices,  or  even 
sets  of  players.  An  optimal  decision  at  each  level,  we  assume,  can  be  obtained  using 
an  optimization  problem. 

Consider  some  of  many  types  of  decisions  made  by  the  computer  processor 
manufacturer  Intel.  First  while  making  the  processor,  manufacturing  errors  and 
uncertainty  can  lead  to  their  “best”  design  being  infeasible.  If  not  infeasible,  the 
design  might  not  be  the  best  choice  under  uncertainty.  This  decision  needs  to  be  made 
accounting  for  the  uncertainty  or  errors  that  can  develop  after  manufacturing  the 
product.  Second,  while  deciding  the  price  (or  quantity)  of  the  processor,  Intel  would 
have  to  take  into  account  what  its  competitors  are  doing  and  if  the  government  has 
made  any  regulations  regarding  taxation  or  distribution.  Setting  a  price,  thus,  not  only 
depends  on  Intef's  own  costs  but  the  strategy  of  other  actors  at  a  different  level  than 
Intel.  Finally,  Intel  needs  to  decide  the  number  of  processors  to  ship  to  specific 
locations.  Even  considering  a  simplified  version  of  the  market  makes  this  a  complex 
problem  as  network  dynamics,  transportation  costs,  and  local  demand  all  weigh  into 
the  decision.  But,  more  importantly,  the  processors  can  only  be  transported  in  positive 
integer  number  quantities,  as  opposed  to  fractional  quantities. 
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All  the  problems  classified  above  fall  under  the  umbrella  of  two-level 
problems.  The  first  decision,  regarding  uncertainty,  requires  the  initial  proposed 
design  of  the  chip  to  be  such  that  the  presence  of  uncertainty  does  not  cause  the 
design  to  be  infeasible  and/or  suboptimal.  The  decision  is  thus  made  to  ensure 
feasibility  of  design  constraints  as  well  as  minimum  variation  in  a  design"s 
perfonnance  under  uncertainty.  Such  a  problem  will  be  described  in  this  dissertation 
as  a  Robust  Optimization  problem. 

The  second  type  of  problem  about  making  a  profit-maximizing  decision  with 
other  players  present  in  a  non-cooperative  competitive  environment  is  known  as  a 
Stacke/berg  Game  in  economics  and  falls  under  the  broad  heading  of  Mathematical 
Programs  with  Equilibrium  Constraints  or  MPECs.  These  problems  have  a  wide 
variety  of  applications,  and  in  their  general  fonn  can  encompass  robust  optimization 
problems  as  well.  A  special  class  of  MPECs  with  certain  mathematical  properties  will 
be  considered  in  this  dissertation  along  with  their  extension  to  Equilibrium  Programs 
with  Equilibrium  Constraints  or  EPECs. 

The  third  problem  is  about  solving  non-cooperative  games  as  well,  except  the 
decision  at  the  second  level  is  to  make  sure  that  the  choice  made  is  integer  rather  than 
continuous.  This  is  more  of  a  computational  issue,  but  nevertheless  the  techniques  to 
solve  such  problems  have  important  applications.  These  problems  fall  into  the  class 
of  Discretely-Constrained  Mixed  Linear  Complementarity  Problems  or  ( DC-MLCPs ). 

The  two  levels  are  a  common  feature  to  all  these  problems,  and  the  biggest 
challenge  to  overcome  this  two-level  structure  is  computational  time.  A  nested 
structure  causes  a  large  increase  in  computational  effort  with  an  increase  in  variables 
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and/or  decision  space  (Bialas  &  Karwan,  1982).  The  focus  of  this  dissertation  is  on 
developing  decomposition  based  solution  techniques  that  reduce  computational  effort 
significantly  for  these  three  types  of  problems.  These  new  techniques  will  then  be 
implemented  on  a  variety  of  examples  from  engineering  and  energy  markets. 

1.2.  Research  Components 

1.2.1.  Solving  Robust  Optimization  Problems 

The  goal  of  robust  optimization  problems  is  to  find  an  optimal  solution  that  is 
minimally  sensitive  to  uncertain  factors.  Uncertain  factors  can  include  inputs  to  the 
problem  such  as  parameters,  decision  variables,  or  both.  Given  any  combination  of 
possible  uncertain  factors,  a  solution  is  said  to  be  robust  if  it  is  feasible  and  the 
variation  in  its  objective  function  value  is  acceptable  within  a  given  user-specified 
range.  Previous  approaches  for  general  nonlinear  robust  optimization  problems  under 
interval  uncertainty  involve  nested  optimization  and  are  not  computationally 
tractable.  The  overall  objective  in  this  dissertation  is  to  develop  an  original  and 
efficient  robust  optimization  method  that  is  scalable  and  does  not  contain  nested 
optimization.  The  proposed  method  is  applied  to  a  variety  of  numerical  and 
engineering  examples  to  test  its  applicability.  Current  results  show  that  the  approach 
is  able  to  numerically  obtain  a  locally  optimal  robust  solution  to  problems  with 
quasiconvex  constraints  (<  type)  and  an  approximate  locally  optimal  robust  solution 
to  general  nonlinear  optimization  problems.  A  portion  of  this  research  component  has 
been  presented  in  (Siddiqui  et  ah,  2011a)  and  (Siddiqui  et  ah,  2011c). 
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1.2.2.  Solving  Mathematical  Programs  and  Equilibrium  Problems  with 
Equilibrium  Constraints 

This  dissertation  presents  an  original  method  for  solving  mathematical  programs  and 
equilibrium  problems  with  equilibrium  constraints  (MPECs  and  EPECs).  Schur"s 
decomposition  followed  by  two  separate  methods  of  approximating  absolute-value 
functions  are  presented  and  used  to  solve  large-scale  MPECs.  The  advantage  of  this 
method  over  traditional  methods  for  solving  MPECs  is  that  computational  time  is 
much  lower,  which  is  corroborated  by  numerical  examples.  An  extension  to  solve 
EPECs  is  also  presented,  along  with  a  small  numerical  example.  Finally,  an 
application  of  the  method  to  an  MPEC  representing  the  United  States  natural  gas 
market  is  given.  A  portion  of  this  research  component  has  been  presented  in 
(Siddiqui  &  Gabriel,  2011b)  and  (Gabriel  et  ah,  2011c). 

1.2.3.  Solving  Discretely-Constrained  Mixed-Integer  Linear 
Complementarity  Problems 

This  research  thrust  presents  an  original  modification  to  a  recent  approach  for  solving 
discretely-constrained,  mixed  linear  complementarity  problems  (DC-MLCPs).  Such 
formulations  include  a  variety  of  interesting  and  realistic  models  of  which  discretely- 
constrained  Nash  games  and  network  equilibrium  problems  are  considered.  A 
methodology  is  provided  to  solve  Nash-Cournot  energy  production  games  allowing 
some  variables  to  be  discrete.  Normally,  these  games  can  be  stated  as  mixed 
complementarity  problems  but  only  pennit  continuous  variables  in  order  to  make  use 
of  each  producer's  Karush-Kuhn-Tucker  conditions.  The  proposed  approach  allows 
for  more  realistic  modeling  and  a  compromise  between  integrality  and 
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complementarity  to  avoid  infeasible  situations.  A  mixed-integer,  linear  program 
formulation  is  used  to  solve  the  DC-MLCP  in  which  both  complementarity  as  well  as 
integrality  are  allowed  to  be  relaxed.  A  portion  of  this  research  component  has  been 
presented  in  (Gabriel  et  ah,  2011a)  and  (Gabriel  et  ah,  2011b). 

1.3.  Organization  of  Dissertation 

The  remainder  of  this  dissertation  is  organized  as  follows.  Chapter  2  provides 
background  and  a  thorough  literature  review  for  the  three  proposed  research 
components.  Chapter  3  provides  the  proposed  solution  methodology  for  robust 
optimization  problems.  The  chapter  also  provides  several  engineering  applications  as 
well  as  numerical  examples.  The  chapter  is  concluded  by  an  example  of  an 
application  to  a  carbon  emissions  related  problem.  Chapter  4  provides  details  on  the 
algorithm  used  to  solve  MPECs  and  EPECs  as  well  as  computational  issues.  The 
chapter  also  provides  numerical  examples  to  corroborate  these  approaches,  as  well  as 
an  application  to  the  North  American  natural  gas  market.  Chapter  5  provides  the 
proposed  solution  technique  for  discretely-constrained  mixed  linear  complementary 
problems  with  examples  of  discretely-constrained  Nash  games  and  energy  networks. 
Chapter  6  provides  conclusions  and  directions  for  future  research.  Figure  1 . 1  displays 
the  organization  of  this  dissertation.  Note  that  the  dashed  line  shows  that  a  technique 
developed  in  Chapter  4  will  be  used  in  Chapter  5 . 
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Figure  1.1:  Organization  of  Dissertation 
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Chapter  2:  Definitions  and  Literature  Review 


2.1.  Introduction 

This  chapter  will  provide  the  necessary  background  for  two-level  optimization 
problems  including  definitions,  tenninologies,  and  a  thorough  literature  review.  This 
chapter  will  initially  give  mathematical  definitions  of  two-level  problems,  and  explain 
how  robust  optimization,  MPECs  and  EPECs,  and  DC-MLCPs  can  all  be  cast  as  two- 
level  problems. 

While  two-level  problems  can  be  shown  to  have  a  general  formulation,  each 
of  the  three  different  types  considered  in  this  dissertation  need  different  treatment  to 
come  up  with  the  most  efficient  solution.  Although  solving  all  three  efficiently  will 
involve  the  use  of  decomposition  techniques,  many  other  alternatives  exist  in  the 
literature  which  will  also  be  discussed.  Finally,  some  preliminary  mathematical  ideas 
and  traditional  algorithms  will  also  be  introduced. 

This  chapter  first  goes  through  the  definition  and  terminologies  used  in  this 
dissertation.  In  particular,  the  next  section  defines  each  of  the  three  two-level 
problems  considered  along  with  other  definitions.  A  literature  review  is  provided  next 
followed  by  two  preliminary  topics. 

2.2.  Definitions  and  Terminologies 

In  general,  the  two-level  optimization  problems  considered  in  this  dissertation  can  be 
expressed  as  the  following 
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min  f(x“,x') 
s.t.(jc“,jc/)  e  Q 
xl  eS(xu) 


(2.1) 


where  the  continuous  variables  x"  e  9C" ,  x'  e  tR"'  are,  respectively,  the  vector  of 
upper- level,  lower-level  variables,  f(xu  ,x‘)  is  the  upper  level  objective  function1,  Q 

is  the  joint  feasible  region  between  these  sets  of  variables  and  S(x“ )  is  the  solution 

□ 

set  of  the  lower-level  problem  that  can  be  an  optimization  problem,  a  nonlinear 
complementarity  problem  (NCP)  (Cottle  et  ah,  2009),  or  a  variational  inequality 
problem  (VI)  (Faccinei  &  Pang,  2003).  Figure  2.1  shows  a  diagrammatic 
representation  of  a  two-level  problem  where  the  nested  structure  is  revealed. 


minimize  f{xu,xl) 

(Upper-Level  Problem) 

/ 

X1 

\ 

Xu 

\ 

/ 

Consider  x11  and  calculate  xl 

infixed’*1) 

(Lower-Level  Problem) 

Figure  2.1:  The  Structure  of  a  Two-Level  Problem 


1  Note  that  when  solving  EPECs,  several  such  two-level  problems  will  be  solved. 
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2.2.1.  Robust  Optimization 


Table  2.1  describes  the  terminology  used  for  robust  optimization. 

Table  2.1:  Definition  of  Terms  for  Robust  Optimization 


Symbol 

Interpretation 

X 

Vector  of  decision  variables 

f 

Objective  function  to  be  minimized 

gj(x,x ) 

Constraint  functions  of  the  form  “<  0” 

Ax 

Maximum  deviations  of  uncertain  variables  from  nominal 

values 

X 

Deviations  from  nominal  values  of  uncertain  variables 

and  parameters,  respectively:  xe[- Ax,+Ax] 

A/0 

User-specified  tolerance  for  acceptable  variation  in 

objective  function  under  uncertainty 

The  goal  in  robust  optimization  is  to  optimize  the  objective  function  with 
respect  to  uncertain  decision  variables  x,  satisfying  all  constraints  and  ensuring  the 
objective  function  variation  is  kept  within  an  acceptable  range  A f0 ,  while  accounting 

for  uncertainty  in  decision  variables.  Specifically,  this  dissertation  considers  robust 
optimization  problems  of  the  form": 


2  Note  that  equality  constraints  are  considered  to  be  formulated  as  two  inequality  constraints  in 
formulation  (2.2).  Alternatively  one  can  explore  the  approach  for  robust  optimization  with  equality 
constraints  (Rangavajhala  et  al.,  2007)  but  that  has  not  been  explored  in  this  dissertation. 
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min  f(x,  x) 


s.t. 

f(x,x)  -  /(x,0)  <  1 

A/0  “  (2.2) 

gj(x,x)<  0  j  =  \,..,J 

x  e  R" ,x  e  Rn“ 

Vx  e  [-Ax,  Ax] 

where  /  and  g  are  continuously  differentiable  in  both  x  and  x .  Figure  2.2 
diagrammatically  shows  the  structure  of  a  robust  optimization  problem. 


Figure  2.2:  Representation  of  a  Robust  Optimization  Problem 

In  the  next  few  paragraphs,  terms  used  in  this  dissertation  are  defined. 

Definition  2.1:  Quasiconvex  Function:  A  function  g(x,x)  is  said  to  be  quasiconvex 
in  x  e  [—  Ax,+Ax]  if  for  all  x  e  [—  Ax,+Ar] ,  gfx,  x)  <  max{g(x,  Ax),  g(x,-Ax)}  for  all  x 
(Bazaraa  et  ah,  1993). 
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Definition  2.2:  Objective  robustness :  For  a  candidate  point  xc  objective  robustness 
holds  if  inequality 

/(xc,x)-/(xc,  0) 

A/0 

is  satisfied  for  all  xe[-  Ax,  Ax] . 


Thus,  this  inequality  ensures  that  the  maximum  objective  function  variation 
stays  below  a  certain  predetermined  maximal  amount  A/0  when  presented  with 
deviations  in  uncertain  variables  and  parameters. 


Definition  2.3:  Feasibility  robustness :  For  a  candidate  solution  xc  if 


gj(xc,x)  <  0  j  = 


is  satisfied  for  all  x  e  [-Ax,  Ax]  then  feasibility  robustness  holds. 


Note  that  equation  (2.3)  is  just  another  constraint,  so  it  can  be  easily 
incorporated  into  inequality  (2.4)  when  stating  a  general  formulation  that  only 
includes  feasibility  robustness.  From  this  point  on,  inequality  (2.3)  will  not  be  stated 
separately  in  any  fonnulation  but  will  be  assumed  to  be  incorporated  in  inequality 
(2.4).  For  a  more  detailed  description  on  objective  robustness,  please  refer  to  (Li  et 
al„  2006). 

Definition  2.4:  Robust  point:  A  robust  point  is  both  objectively  and  feasibly  robust. 
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Definition  2.5:  Locally  optimal  robust :  For  a  robust  optimization  problem,  a  locally 
optimal  robust  solution  x*,  is  a  robust  point  such  that  there  exists  a  neighboring  set  U 
of  robust  solutions  for  which  x*  is  optimal  (  f  (x*)  <  / (x),  Vx  e  U  ). 

It  is  essential  that  the  neighboring  set  be  made  up  of  only  robust  points 
otherwise  the  term  is  ill-defined.  There  is  also  a  global  counterpart  as  defined  below. 

Definition  2.6:  Globally  optimal  robust :  For  a  robust  optimization  problem,  a 
globally  optimal  robust  solution  x*,  is  a  robust  point  such  that  x*  is  optimal 
(  f(x*)  <  /(x),  Vx)  in  the  feasible  region. 


2.2.2.  Mathematical  and  Equilibrium  Programs  with  Equilibrium 
Constraints 

3 

In  general,  a  mathematical  program  with  equilibrium  constraints  is  given  by 

min/(x,  y) 

s.t.(x,y)eQ  (2.5) 

y  e  S(X) 

where  the  continuous  variables  x  e  ,  yeSR"'  are,  respectively,  the  vector  of 

upper- level,  lower-level  variables,  f(x,y )  is  the  upper-level  single-objective 

□  U 

□ 

3  Without  loss  of  generality,  we  assume  that  the  variables  x  and  y  are  nonnegative,  which  is 
incorporated  in  the  decision  space  Q. 
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function,  Q  is  the  joint  feasible  region  between  these  sets  of  variables  and  S(x)  is  the 
solution  set  of  the  lower-level  problem  that  can  be  an  optimization  problem,  a 
Nonlinear  complementarity  problem  (NCP),  or  variational  inequality  problem  (Luo  et 
al„  1996). 

One  focus  of  this  dissertation  is  when  S(x )  is  a  solution  to  a  nonlinear 
complementarity  problem.  Having  a  function  g :  9?"  — >  9?" ,  a  nonlinear 
complementarity  problem  is  to  find  a  vector  z  e  51"  such  that: 

z  >  0 

g(z)>On  (2.6) 

zrg(z)  =  0 

If  S(x )  is  the  solution  set  of  an  NCP,  (2.5)  can  be  rewritten  as 

min  f(x,y) 
s.t.(x,y)  e  Q 

L  -  0  (2.7) 

g(x,y)>0 

yTg(x,y)  =  0 

where  g(x,  y) :  91”'  x  91" 1  — >  91  "J'  is  a  vector-valued  function. 

Similarly,  an  EPEC  is  defined  as  a  game  between  N  players  at  the  top  level 
where  each  top-level  player  solves  an  optimization  problem  of  the  form  (2.7).  Hence, 
an  EPEC  with  a  common  lower-level  for  each  of  the  N  upper-level  players  typical  of 
Stackelberg  leaders  in  energy  production  with  the  rest  of  the  market  represented  by 
the  lower-level  problem  is  given  by 
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j  = 


min  fj(x,y) 
s.t^x,^)  e  Q 

y>0  (2.8) 

g(x,y)>  0 
yTg(x,y)  =  o 

Figure  2.3  shows  the  diagrammatic  representation4  for  an  MPEC  and  Figure  2.4 
shows  the  diagrammatic  representation  for  an  EPEC. 


4  Nash-Cournot  in  this  diagram  implies  that  an  individual  player  solves  their  own  optimization  problem 
with  other  players"  decisions  being  fixed. 
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maximize  Profit(x,y ) 


(Decides  the  value  of  x) 


Figure  2.3:  Representation  of  an  MPEC 


Nash-Cournot 

y fixed) 

(Decides  the  value  of  x) 


Nash-Cournot 

(y fixed?  y) 

(Observe  x  and  solve  for  y) 
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2.2.3  Discretely-Constrained  Mixed  Linear  Complementarity  Problems 

It  is  not  immediately  obvious  why  the  problem  considered  in  this  subsection  is  a  two- 
level  problem.  The  problem  in  its  original  fonn  is  not,  but  it  needs  to  be  converted 
into  a  two-level  fonn  for  the  particular  solution  technique  (Gabriel  et  ah,  2011a; 
Gabriel  et  al,  2011b)  to  be  applicable.  In  general,  a  discretely-constrained  mixed 
linear  complementarity  problem  is  given  as  follows:  given  the  vector  q=(qx,q2)T 


and  matrix  A  = 


,  find  z  =  (z],z2)t  e  9?"1  x 01  "2  such  that 5 


0<<7,+Ui 

\Z2j 

0  =  q2+(A2X  A22)Z>  I  T  z2,z2  free  (2.9) 

lZ2  ) 

(zj)c  em+,ceC1,(z1)d  cZ+,d  ei), 

(z2)c  e  s-h+,c  e  C2,(z2)d  e  Z+,d  e  D2 

The  indices  for  z,-,  i  =  1,  2  are  partitioned  into  continuous-valued  (denoted  by 
the  set  Cj)  and  discrete-valued  variables  (denoted  by  the  set  D,),  i.e., 

z,  =  ((z;.  Jc  ,  (z;  J ,  /  =  1 ,2  with  the  continuous  variables  shown  first  without  loss  of 

generality.  From  here  on,  unless  otherwise  indicated,  the  discrete  sets, 
Z)j  =  {0,l,...,iV}  and  D2  =  {—  ,...,— 1,0,1,. ..,N2}  will  be  assumed  with  N,  N\,  N2, 
nonnegative  integers. 


5  Here  the  superscript  T  denotes  the  transpose  function.  The  symbol  T  denotes  complementary  which 
means  that  the  product  of  the  two  terms  must  be  zero. 
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Finding  a  solution  to  this  DC-MLCP  can  be  thought  of  as  a  two-level 
problem,  even  though  (2.9)  fonnulates  it  in  one  level.  The  upper  level  minimizes 
deviations  from  an  integer  solution  and  complementary,  i.e.,  ensures  that  as  close  as 
possible  to  an  integer  solution  is  obtained  while  satisfying  complementary  conditions 
with  a  minimum  deviation  as  well,  while  the  lower  level  solves  a  complementary 
problem  assuming  some  deviation  from  integers  has  been  fixed  at  the  upper  level. 
Figure  2.5  shows  the  diagrammatic  representation  of  a  discretely-constrained  mixed 
linear  complementary  problem,  while  the  following  formulation  describes  the  two- 
level  fonnulation.  Note  that  the  first  two  inequality  constraints  and  the  first  equality 
constraint  (the  first  three  constraints)  form  a  complementary  problem.  Hence,  the 
two-level  structure6  is  apparent  in  the  following  formulation.  Chapter  5  will  describe 
in  detail  how  this  two-level  fonnulation  is  obtained. 


1  Compare  (2.10)  to  (2.1).  The  upper-level  variables  are  e  and  i.e.,  _  f  £  }  and  the  first  three  lines 


a 


in  (2.10)  define  the  upper-level  problem.  The  lower  level  variables  z,  have  J  _ 


and  are  part  of 


the  solution  set  of  the  discretely-constrained  complementary  problem  given  by  the  last  four  lines  of 
(2.10). 
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mm(lr£  + 1  rcr) 
s.t. 


(a  \ 

1  e  91,  <f  e  Dx  uD, 

Vf2 


Me  eSR^ceCj 


+  (^ll  ^12  j 


■27 


0  C^2  7“  (^21  ^22 


-  cr  _L  (zj  -  cr)  =  0 


_!_  z2 ,  z2  free 


(Zl)c  em^ceC^izJ, -(sl)deZ+,deDl 
{z2)c  e91  +  ,ceC2,(z2)rf  -{s2)d  eZ+,deD2 


(2.10) 
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minimize  f(s,£,z) 


(Minimize  deviations  s,C) 


Figure  2.5:  Representation  of  a  Discretely-Constrained  Mixed  Linear 

Complementary  Problem 


2.3.  Overview  of  Previous  Work 


2.3.1  Robust  Optimization 

This  dissertation"s  approach  for  solving  robust  optimization  problems  (hereafter 
referred  to  as  the  modified  Benders  method),  which  will  be  described  in  Chapter  3, 
will  now  be  compared  to  previous  methods.  A  comprehensive  review  of  the  literature 
was  conducted  and  the  main  distinctions  between  the  proposed  modified  Benders 
method  and  previous  works  are  presented  as  follows. 

The  robust  optimization  problems  in  the  proposed  modified  Benders  method 
also  involve  nonlinear  (for  example,  Welded  Beam  and  Heat  Exchanger,  which  both 
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involved  nonconvex  constraint  functions)  constraint  functions7 8.  This  is  more  general 
than  only  considering  linear  constraint  functions  in  the  problem  as  reported  in  the 
literature  (e.g.,  Balling  et  ah,  1986;  Ben-Tal  &  Nemirovski,  2002;  Bertsimas  &  Sim, 
2006;  Soyster,  1973)  or  quadratic  (e.g.,  Li  et  ah,  2011)  as  well  as  other  versions 
involving  convex  programs  (e.g.,  Ganzerli  &  Pantelides,  1999)  or  linearization  to 
solve  the  problem  (e.g.,  Balling  et  ah,  1986).  The  modified  Benders  method  is  able  to 
obtain  exact  locally  optimal  robust  solutions  to  problems  with  quasiconvex 
constraints  as  well  as  non-convex  quadratic  programs,  which  no  one  method  in  the 
reported  literature  is  able  to  achieve.  Other  approaches  also  consider  distributions  for 
uncertainty  (e.g.,  Lee  et  ah,  2009;  Lagaros  &  Papadrakakis,  2007)  while  the  approach 

o 

of  this  dissertation  looks  at  a  worst-case  analysis  for  interval  uncertainty  without  any 
explicit  probability  distribution  or  a  nested  optimization  structure.  Moreover,  the 
modified  Benders  method  is  able  to  handle  large  uncertainties  which  earlier  methods 


7  In  some  cases,  although  not  considered  in  this  dissertation,  a  slightly  stricter  condition  with  convexity 
in  the  lower-level  of  the  Benders  decomposition  method  is  needed.  However,  we  did  not  encounter  this 
in  any  of  our  test  problems.  A  workaround  to  this  problem  is  available  in  (Gabriel  et  at,  2010).  This 
involves  sampling  the  domain  of  the  objective  function  of  the  lower-level  optimization  problem  to 
determine  the  convex  portions  of  this  function.  This  numerical  approximation  scheme  can  be  applied  to 
the  modified  Benders  method  to  determine  convexity  of  the  objective  function  of  the  lower-level 
optimization  problem. 

8  Note  that  this  dissertation  considers  robust  optimization  problems  with  interval  uncertainty,  while 
there  is  a  substantive  amount  of  literature  considering  other  types  of  uncertainty.  Refer  to  (Bertsimas  & 
Sim,  2006;  (Ben-Tal  et  al.,  2009). 
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(e.g.,  Balling  et  al.,  1986;  Soyster,  1973;  Ganzerli  &  Pantelides,  1999)  were  not  able 
to  tackle. 

The  proposed  approach  preserves  the  computational  tractability,  theoretically 
and  practically,  of  the  deterministic  (i.e.,  nominal)  problems.  By  contrast,  under 
interval  uncertainty,  the  computational  effort  for  previous  methods  (e.g.,  Gunawan  & 
Azarm,  2004;  Li  et  al.,  2006)  to  obtain  robust  solutions  is  much  higher  than  their 
detenninistic  counterparts.  However,  results  from  a  variety  of  numerical  experiments 
show  that  the  computational  effort  of  solving  the  robust  optimization  problems  is  not 
much  greater  than  that  of  their  deterministic  counterparts  for  the  modified  Benders 
method.  Moreover,  the  modified  Benders  method  is  scalable,  in  that  by  numerical 
tests,  the  number  of  function  calls  per  iteration  increases  at  most  linearly  (numerical 
result)  with  an  increase  in  the  number  of  variables,  uncertainty  variables,  and 
constraints. 

Since  this  dissertation"s  approach  is  based  on  gradient-based  methods,  a 
globally  optimal  robust  solution  can  never  be  guaranteed  for  the  complete  class  of 
continuous,  non-convex  problems.  However,  this  dissertation  uses  the  idea  of  a 
locally  optimal  robust  solution,  and  shows  that  this  approach  can  obtain  a  locally 
optimal  robust  solution  for  nonlinear  robust  optimization  problems. 

In  addition  to  the  uncertainty  in  the  data  of  the  problems  (i.e.,  the  parameters), 
interval  uncertainty  is  considered  in  the  decision  variables  corresponding  to 
manufacturing  tolerances,  implementation  errors,  etc.  where  optimized  values  cannot 
be  achieved  exactly,  which  is  very  common  in  practical  engineering  applications.  For 
the  current  robust  optimization  fonnulations  in  the  literature  (e.g.,  Ben-Tal  & 
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Nemirovski,  2008;  Lu  et  al,  2010;  Qiu  &  Wang,  2010;  Zhu  &  Ting,  2001), 
considering  uncertainty  in  the  decision  variables  may  considerably  change  those 
formulations  or  increase  the  complexity  of  the  problem.  The  approach  in  this 
dissertation,  however,  keeps  the  same  formulation  and  obtains  locally  optimal  robust 
solutions  to  these  problems  with  not  much  greater  computational  effort  than  the 
detenninistic  problem. 

There  has  been  an  abundance  of  literature  modifying  Benders  decomposition 
method  (Benders,  1962)  to  solve  various  types  of  optimization  problems.  However,  to 
my  knowledge,  there  have  not  been  any  modifications  to  Benders  method  that  solve 
nonlinear  robust  optimization  problems  with  interval  uncertainty  although  Benders- 
based  robust  optimization  problems  have  been  considered  in  other  contexts.  For 
example,  (Velarde  &  Laguna,  2004)  provided  a  Benders-based  heuristic  to  solve  the 
international  source  allocation  problem.  In  this  problem,  a  subset  of  international 
suppliers  needs  to  be  selected  to  meet  local  demand.  The  uncertainty  is  in  the  demand 
function  parameters  and  exchange  rates.  However,  their  approach  did  not  consider 
uncertainty  in  variables.  For  their  approach  to  work,  they  needed  to  include  control 
variables,  which  change  depending  on  the  uncertainty  scenario  to  provide  an  easier 
route  to  solution.  The  approach  in  this  dissertation  does  not  require  the  introduction  of 
such  variables.  Also,  their  methodology  can"tbe  extended  to  general  nonlinear  robust 
optimization  problems.  Saito  and  Murota  (Saito  &  Murota,  2007)  described  a  method 
to  apply  Benders  decomposition  to  solve  linear,  mixed-integer,  robust  optimization 
problems  with  ellipsoidal  uncertainty.  However,  this  approach  only  works  for  linear 
problems.  Finally,  Montemanni  (Montemanni,  2006)  applied  a  Benders  algorithm  to  a 
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specific  robust  spanning  tree  problem,  while  Ng  et  al.  (Ng  et  al.,  2010)  applied  it  to  a 
specific  semiconductor  allocation  problem  that  had  uncertainty.  Again,  both 
approaches  are  not  applicable  to  continuous,  nonlinear  robust  optimization  problems 
with  interval  uncertainty  and  have  not  modified  Benders  decomposition  in  the  way 
this  dissertation  does. 

There  are  related  topics  to  robust  optimization  such  as  anti-optimization  (e.g., 
(Qiu  &  Wang,  2010)  and  reliability-based  design  optimization  (e.g.,  Zou  & 
Mahadevan,  2006)  that  run  into  the  same  problems  as  described  above  of  not  being 
computationally  tractable  or  only  working  for  a  certain  simple  type  of  problems.  For 
example,  Youn  and  Xi  (Youn  &  Xi,  2009)  modified  a  double  loop  problem  (like 
robust  optimization)  into  a  single  loop  so  that  it  becomes  computationally  easier.  This 
work  involves  using  an  eigenvector  dimension  reduction  method,  and  probability 
distributions,  which  may  not  be  applicable  to  general  nonlinear  robust  optimization 
problems.  Also,  neither  of  these  papers  has  techniques  that  include  interval 
uncertainty  in  parameters,  in  decision  variables,  along  with  being  computationally 
efficient.  The  modified  Benders  method  of  this  dissertation  is  not  only  directly 
relevant  to  robust  optimization,  but  it  handles  the  specific  two-level  structure  of 
robust  optimization  in  a  less  computationally  intensive  way. 

2.3.3  Mathematical  and  Equilibrium  Programs  with  Equilibrium 

Constraints 

Finding  optimal  points  for  mathematical  programs  with  equilibrium  constraints 
(MPECs)  involves  solving  a  two-level  optimization  where  the  lower  level  is  an 
equilibrium  problem.  In  particular,  having  a  complementarity  problem  (Cottle  et  al., 
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2009)  as  the  lower  level  implies  that  the  complementarity  constraint  is  a  non-convex 
bilinear  multiplicative  term. 

Many  techniques  exist  to  solve  MPECs  (Luo  et  al.,  1996)  but  a  popular  way 
such  MPECs  have  been  solved  is  by  using  a  disjunctive-constraints  technique 
(Fortuny-Amat  &  McCarl,  1981).  However,  the  two  biggest  drawbacks  of  disjunctive 
constraints  are  that  the  method  is  computationally  expensive  for  large  models  (Luo  et 
al.,  1996)  and  that  selecting  a  particular  constant  in  the  method  is  often  troublesome 
(Gabriel  &  Leuthold,  2010).  The  solution  can  be  extremely  sensitive  to  the  selection 
of  this  constant,  and  be  far  from  the  true  answer  if  not  selected  correctly.  Other 
methods  (Steffensen  &  Ulbrich,  2010)  and  (Uderzo,  2010)  also  exist  but  have  not 
been  shown  to  work  for  large-scale  models. 

This  dissertation  presents  a  new  method  for  solving  MPECs,  based  on 
handling  the  bilinear,  non-convex  tenn  using  Schur's  Decomposition  and  Special 
Ordered  Sets  of  Type  1  (SOS  Type  1)  variables  (Gabriel  et  al.,  2006),  along  with  a 
refonnulation  technique  for  absolute  value  terms.  This  method  is  applied  to  solve  a 
small  Stackelberg  game  with  the  number  of  players  allowed  to  vary  and  an  MPEC  for 
the  U.S.  natural  gas  market  to  validate  the  proposed  approach.  A  proposed  extension 
along  with  a  simple  example  to  solve  equilibrium  programs  with  equilibrium 
constraints  (EPECs)  is  also  provided. 

2.3.4.  Discretely-Constrained  Mixed  Linear  Complementarity  Problems 

As  discussed  before,  complementary  problems  have  had  several  applications  in  the 
literature,  including  solving  Nash-Cournot  games  and  network  problems.  Both  Nash- 
Cournot  games  and  network  problems  can  be  converted  to  mixed  complementary 
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problems  by  taking  the  Karush-Kuhn-Tucker  (KKT)  (Bazaraa  et  al.,  1993)  conditions 
to  each  player's  optimization  problem  and  combining  them  (Cottle  et  al.,  2009). 

A  lot  of  applications  of  both  these  problems  relate  to  energy  markets.  For 
example,  Bard  (1983,1988)  developed  algorithms  for  linear  and  convex  two-level 
programming  problems  with  applications  to  energy.  Continuing,  Bard  and  Moore 
(Bard  &  Moore,  1990)  introduced  a  branch  and  bound  algorithm  for  two-level 
problems  resulting  from  complementary  problems.  Karlof  and  Wang  (Karlof  & 
Wang,  1996)  applied  a  two-level  approach  to  solving  a  flow  shop  scheduling 
problem,  while  (Labbeb  et  al.,  1998)  applied  it  to  a  model  of  taxation  and  highway 
pricing.  Moore  and  Bard  (Moore  &  Bard,  1990)  and  Wen  and  Huang  (Wen  &  Huang, 
1996)  provided  methods  to  solve  mixed-integer  two-level  problems,  but  these 
methods  are  not  applicable  for  solving  DC-MLCPs. 

More  recently  (Bard  et  al.,  2000),  (Fuller,  2010),  (Fuller,  2008),  (Gabriel  & 
Leuthold,  2010),  (Gabriel  et  al.,  2010),  (Hu  et  al.,  2009),  (Marcotte  et  al.,  2001), 
(O'Neill  et  al.,  2005),  and  (Scaparra  &  Church,  2008)  have  had  applications  of  game 
theory  problems  to  energy  but  none  has  considered  complementary  problems  which 
are  discretely  constrained. 

In  some  cases,  solutions  to  discretely-constrained  complementary  problems  do 
not  exist.  This  is  because  satisfying  integrality  and  complementary  conditions 
together  can  prove  to  be  more  difficult  than  satisfying  integrality  and  complementary 
conditions  individually.  However,  solutions  to  the  original  optimization  problem 
whose  KKT  conditions  were  used  to  formulate  the  complementary  problem  might 
still  exist. 
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(Gabriel  et  al,  2011a),  (Gabriel  et  al,  2011b)  provide  ways  around  this,  and 
provide  a  relaxation  technique  that  is  able  to  solve  DC-MLCPs.  The  distinguishing 
features  of  the  proposed  technique  in  both  these  papers  with  respect  to  other 
procedures  reported  in  the  technical  literature  (e.g.,  (Galiana  et  al.,  2003),  (Hogan  & 
Ring,  2003),  (Bjorndal  &  Jomsten,  2008)  are  two-fold.  First,  the  initial  Nash-Cournot 
game  or  network  problem  is  not  manipulated  to  achieve  prices  that  support  market 
outcomes.  Instead,  optimality  conditions  of  the  original  problem,  with  integrality  and 
complementary  conditions  relaxed,  are  formulated  and  incorporated  into  a  relaxation 
problem  that  allows  realizing  the  tradeoff  of  integrality  vs.  complementarity.  Second, 
instead  of  using  a  two-step  procedure  as  in  the  literature,  the  technique  in  (Gabriel  et 
al.,  2011a)  is  single-step  fonnulation  of  a  two-level  problem,  and  does  not  require 
altering  the  original  problem  by  fixing  integer  variables  to  their  optimal  values  to 
formulate  a  continuous  problem.  Hence  this  dissertation  will  concentrate  on  the  work 
from  the  two  papers  (Gabriel  et  al.,  2011a),  (Gabriel  et  al.,  2011b). 

In  both  papers,  (Gabriel  et  al.,  2011a),  (Gabriel  et  al.,  2011b),  however, 
disjunctive  constraints  (Fortuny-Amat  &  McCarl,  1981)  are  used  to  solve  the 
resulting  two-level  problem.  In  this  dissertation,  a  method  developed  in  Chapter  4 
will  be  used  to  solve  these  problems  instead  of  disjunctive  constraints.  This  is  the 
main  contribution  in  this  dissertation,  in  that  the  method  of  Chapter  4  solves  the  DC- 
MLPCs  much  quicker  than  the  disjunctive  constraints  method,  and  does  not  require 
the  selection  of  a  specific  constant  for  disjunctive  constraints. 
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2.4.  Preliminaries 


2.4.1.  Benders  Decomposition 

This  section  describes  standard  Benders  decomposition  as  a  modified  version  will  be 
used  in  Chapter  3.  Table  2.2  describes  the  terminology  used  for  explaining  Benders 
decomposition. 


Table  2.2:  Definition  of  Terms  for  Benders  Decomposition 


Symbol 

Interpretation 

vc 

Complicating  vector  of  variables  to  explain  standard  Benders 

decomposition 

Vu 

Uncomplicating  vector  of  variables  to  explain  standard 

Benders  decomposition 

c(vc) 

Constraints  of  complicating  variable 

d(vc,vu) 

Constraints  of  uncomplicating  and  complicating  Variables 

Benders  decomposition  is  used  to  efficiently  solve  linear  and  nonlinear 
programs  (Conejo  et  ah,  2006)  and  decomposes  the  original  set  of  variables  into  both 
complicating  and  uncomplicating  ones.  Normally,  integer  variables  are  defined  as 
complicating  variables  as  fixing  their  values  allows  the  problem  to  have  a  structure 
that  provides  an  easy  solution  (assuming  the  rest  of  the  problem  is  relatively  easy  to 
solve).  However,  in  general,  the  complicating  variables  need  not  be  integer  and  can 
be  real-valued  which,  if  fixed,  render  a  simple  or  decomposable  problem.  Using 
notation  provided  above,  Benders  decomposition  seeks  to  solve  an  optimization 
problem  of  the  following  fonn: 
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min  f(vc,vu) 

vc,vu 

s.t. 


c(vc)<0 
d(vc,vu)<  0 
where 
vcet/ci R" 

V„£9T 


(2.10) 


c(vc):tH"  ->9F 
d(vc,vu):dr/m  ->9?? 

To  solve  this  problem,  the  Benders  decomposition  technique  fixes  values  of 
vc  (which  are  part  of  a  set  U  that  can  be  integers  or  other  subsets  ofTT)  and  solves 
the  problem  after  first  decomposing  into  a  master  problem  and  sub-problem  (Conejo 
et  ah,  2006).  To  explain  these  notions  of  master  and  sub-problem,  first  define  an 
auxiliary  function  a(vc)  as  follows  which  expresses  the  objective  function  of  the 
original  problem  as  a  function  solely  of  the  complicating  variables. 

a(vc)  =  mmf(vc,vu) 

vu 

s.t.  (2.11) 

d{vc,vu)<  0 


Using  the  definition  of  a(vc),  the  original  problem  (2.10)  can  be  expressed  as  follows. 

min  a{  v  ) 

vc 

s.t.  (2.12) 

c(vc)<0 

Iteratively,  a  subproblem  (2.13)  is  solved  to  approximate  a( vc)  from  above  by 
fixing  values  of  the  complicating  variables  (vc  =  v/xed )  and  obtaining  the  dual 
variables  X  to  these  constraints  as  shown  in  (2.13). 
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min  f(vc,vu) 


vu 

s.t. 


vc  =  v/,xed  ( dual :  A) 

^(VC’V»)  -  0 


(2.13) 


Then,  the  solution  to  the  above  problem,  (v/'"  )  and  the  dual  variables  (/Lso/) 

are  used  to  construct  “Benders  cuts”  in  the  master  problem  to  approximate  the 
function9  a(vc )  from  below.  Note  that  these  cuts  are  iteratively  added  at  each  step 
until  convergence.  For  simplicity,  only  one  cut  is  shown  here. 

min  a 

a,vc 


S.t. 

c(  vc)  <  0 

a  >  ,  vf)  +  (r‘  J  (v„  -  vf -  ) 


(2.14) 


Since  the  master  problem  (2.14)  has  a  larger  feasible  region,  it  provides  a 
lower  bound  (z/c)  while  the  more  restricted  subproblem  (2.13)  provides  an  upper 
bound  (zup)  for  the  solution  objective  function  value.  These  problems  are  solved 
iteratively  until  ( zup  -  z/0)/(  z/c)  is  less  than  some  tolerance.  As  long  as  the  function 
a(vc)  is  convex,  Benders  decomposition  converges  to  an  optimal  solution  (Conejo  et 
al„  2006). 


9  A  sufficient  condition  for  convergence  is  that  the  objective  function  in  formulation  (2.11)  needs  to  be 
convex.  However,  the  modification  this  paper  presents,  from  the  experimental  results,  does  not  need 
this  convexity  as  the  Benders  cuts  are  modified.  For  more  information,  please  refer  to  (Conejo  et  al., 
2006)  and  (Gabriel  et  al.,  2010). 
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2.4.2.  Disjunctive  Constraints 


From  (2.7),  the  set  of  solutions  to 

yrg(*,y)  =  o  (2-15) 

is  nonconvex  and  can  be  computationally  challenging  to  find  even  if  g(x,  y )  is  linear. 

One  way  is  to  use  disjunctive  constraints  (Fortuny-Amat  &  McCarl,  1981).  A 

large  constant  K  is  introduced,  which  can  be  difficult  to  select  and  cause 

computational  issues  (Gabriel  &  Leuthold,  2010),  as  well  as  a  vector  of  binary 

variables  r.  Then,  (2.8)  is  rewritten  as 

min  f(x,y) 
s.t.(x,y)  e  Q 
0<y<K(\-r) 

0  <g(x,y)<Kr  (2.16) 

where 

r  £  {0.1}’’  is  a  vector  of  binary  values 
K  e  9f++  is  a  large  constant 

For  large  enough  K,  the  solution  set  to  (2.16)  is  equivalent  to  that  of  (2.7).  The 
binary  vectors  and  large  K  force  componentwise,  at  least  one  of  y  or  g  to  be  0. 
However,  choosing  K  too  small  can  cause  errors  in  problem  formulation  (Gabriel  & 
Leuthold,  2010)  while  choosing  K  too  large  can  cause  the  condition  number  of  the 
optimization  problem  (Renegar,  1995),  (Renegar,  1994)  to  be  high  and  result  in 
numerical  errors.  One  of  the  main  aims  of  this  section  is  to  get  around  this  problem 
by  using  decomposition  and  approximation  techniques. 


30 


2.4.3.  Approximating  Nonlinear  Functions  using  SOS  Type  1  and  Type  2 
Variables 

Note  that  disjunctive  constraints  are  used  to  state  disjunctive  (either/or)  logic 
statements.  Hence,  Chapter  4  will  provide  a  different  way  to  state  such  logic 
statements  mathematically.  For  this,  a  refonnulation  of  the  absolute  value  function 
will  be  required,  for  which  special  ordered  sets  will  be  used. 


Definition  2.7:  A  Special  Ordered  Set  of  Type  One  (SOS1  or  SOS  Type  1)  is 
defined  to  be  a  set  of  non-negative  variables  for  which  at  most  one  member  from 
the  set  may  be  non-zero  in  a  feasible  solution.  There  are  no  other  restrictions  on  the 
elements  of  the  set,  and  they  can  be  ordered  in  any  way. 


Among  the  uses  for  SOS1  variables,  one  popular  one  is  to  approximate 
functions.  For  example,  consider  a  nonlinear  function  g(x)  over  a  closed  interval 
x^[xlow,xhigh\  on  the  positive  real  line.  Given  a  partition  of  n  points  of  the 

interval  x(.  e  {xtow  =  jcj,jc2,jc3,...,jcn  =JcAi  A},  a  new  SOS  Type  1  set  of  n  variables 

{v,  }"=1  can  be  introduced  to  approximate  this  nonlinear  function.  Then,  g(x)  can  be 
expressed  as 
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g(x)  =  YjViS(xi) 

i= 1 

where 

x  =  Yj  vtxi  (2-17) 

1=1 

n 

Z>.- 1 

1=1 

{v(.}"=1  are  SOS  1  Variables 

Figure  2.6  shows  this  nonlinear  function  being  approximated  by  SOS  Type  1 
variables. 


g(x) 


A  /  . V  i  .V,  ^ 


Figure  2.6:  Approximating  a  Nonlinear  Function  Using  SOS  Type  1  Variable 
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Similarly,  a  piecewise  approximation  can  also  be  developed  using  SOS  Type 


2  variables. 


Definition  2.8:  A  Special  Ordered  Set  of  type  Two  (SOS  2  or  SOS  Type  2)  is  a  set 
of  nonnegative  consecutive  variables  in  which  not  more  than  two  adjacent 
members  may  be  non-zero  in  a  feasible  solution.  No  other  restrictions  are  placed 
on  the  set. 


Again,  consider  a  nonlinear  function  g(x)  over  a  closed  interval 
x  e  \_xlow,xhiA  on  the  real  line.  Given  a  partition  of  n  points  of  the 

interval  xt  e  {xlow  =  xl,x2,x3,...,xn  =xhi  l},  a  new  SOS  Type  2  set  of  n  variables 

{uj  }"=1  can  be  introduced  to  approximate  this  nonlinear  function.  Then,  g(x)  can  be 
expressed  as 

n 

g(x)  =  YJuig(xi) 

i= 1 

where 

x  =  YjU,xi  (2-18) 

i= 1 
n 

2>,=1 

1=1 

{u[  }"=1  are  SOS  2  Variables 

Figure  2.7  shows  this  nonlinear  function  being  approximated  by  SOS  Type  2 
variables.  The  red  line  shows  the  piecewise  linear  approximation  of  the  function. 
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g(x) 


Figure  2.7:  Approximation  of  Nonlinear  Functions  using  SOS  Type  2 

Variables 

The  downside  of  SOS  Type  2  variables  is,  of  course,  that  it  requires  much 
more  computational  power  than  if  SOS  Type  1  variables  were  used.  In  Chapter  4, 
however,  an  absolute  value  function  will  be  used.  Hence,  setting  g(x )  =  |x|.  For  this 
purpose,  only  a  set  of  two  SOS  Type  1  variables  is  required  to  reformulate  this  over 
the  entire  range.  This  is  described  by  the  following  formulation. 


g(x)  =  v+  -v 
where 
X  =  v+  + 

v+,v  arcSOSTypcl  Variables 


(2.19) 
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Since  these  variables  encompass  the  whole  range,  no  restriction  on  the  sum 
being  1  is  required.  Moreover,  this  formulation  will  be  used  in  Chapter  4  to  solve 
MPECs. 
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Chapter  3:  Solving  Robust  Optimization  Problems  Using  a 
Modified  Benders  Method 

3.1.  Introduction 

Engineering  optimization  problems  often  involve  uncontrollable  variations  or 
uncertainties  in  factors  like  decision  variables  and/or  parameters.  Optimal  solutions 
that  might  be  deterministically  feasible  often  end  up  being  infeasible  for  a  given 
realization  of  uncertain  factors.  Additionally,  even  small  levels  of  variations  can 
cause  large  degradations  in  the  objective  function  value.  Manufacturing  errors, 
measurement  problems,  and  uncertainty  in  environmental  conditions  are  examples  of 
sources  for  these  variations. 

Uncertainty  can  be  handled  with  or  without  a  probability  distribution. 
Optimization  problems  that  involve  probability  distributions  are  referred  to  as 
stochastic  optimization  problems.  These  are  more  suited  for  situations  where 
accounting  for  worst-case  uncertainty  might  result  in  foregoing  performance. 
Optimization  problems  in  this  chapter  are  more  suited  for  situations  where  any 
violation  of  constraints  under  uncertainty  could  result  in  the  solution  being  unsuitable. 
Hence,  a  worst-case  analysis  needs  to  be  appropriate  for  problems  considered  in  this 
chapter. 

In  this  section,  an  approach  for  robust  optimization,  e.g.,  (Ben-Tal  et  ah, 
2009),  for  linear,  quadratic,  convex,  and  non-convex  programs  is  developed  by 
applying  a  worst-case  analysis  using  a  decomposition  method.  No  probability 
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distribution  is  presumed10,  but  only  intervals  with  a  nominal  point  (user-  or  problem- 
defined)  are  used  to  represent  the  uncertainty  in  decision  variables  and/or  parameters. 
The  problem  structure  in  this  dissertation  reflects  a  real-world  design  situation,  e.g., 
when  information  about  uncertain  factors  during  the  early  stages  of  a  design  process 
is  often  limited. 

The  two-level  structure  is  apparent  in  robust  optimization  problems.  The 
upper-level  of  a  robust  optimization  problem  is  a  decision  based  on  a  fixed  level  of 
uncertainty.  The  lower-level  checks  the  feasibility  of  an  optimal  solution  obtained 
from  the  upper-level.  This  chapter  provides  a  way  to  decompose  this  two-level 
structure  using  Benders  decomposition  to  solve  the  robust  optimization  problem.  A 
portion  of  the  material  in  this  chapter  has  been  presented  previously,  see  (Siddiqui  et 
al.,  2011a)  and  (Siddiqui  et  ah,  2011c). 

3.2.  Interval  Uncertainty 

A  simple  example  will  be  presented  first  to  motivate  this  method.  Consider  the 
optimization  problem 


10  The  statement  that  no  probability  distribution  being  presumed  is  to  ensure  the  fact  that  probability 
does  not  come  into  play  in  any  part  of  the  discussed  formulation.  For  instance,  a  uniform  distribution 
over  the  whole  interval  of  uncertainty  can  be  assumed.  However,  the  solution  technique  for  robust 
optimization  would  involve  solving  the  problem  while  ensuring  there  is  a  zero  probability  of  constraint 
violation,  thus  taking  probability  out  of  the  question.  Therefore,  it  is  informative  to  presume  no 
probability  distribution. 
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min  / (x)  =  — X|  -  2x2 


s.t. 

gx=xx+x2<  8  (3.1) 

g2  =  -2xj  +  lx,  <  5 
g3  =  -Xj  -3x2  <  -10 

A  version  of  this  problem  with  uncertainty  looks  like 

min  / (x)  =  -Xj  -  2x2 
s.t. 

gi  ^(l  +  iOxj  +(l  +  x2)x2  <8 

g2  =(-2  +  x3)x! +(l  +  x4)x2  <5  (3.2) 

g3  =(-l  +  x5)Xj  +(-3  +  x6)x2  <-10 
where 

Vx;.  e  [- 0.1,+0.l]  i  =  l,. ..,6 

Note  that  parameter  uncertainty  has  been  introduced  in  the  constraints  of  the 

problem.  Realize  also  that,  for  example,  if  x,  =  x,  =0.1  in  the  first  constraint  of  (3.2), 

then  if  x\  and  X2  satisfy  the  following  inequality 

(1  +0. l)x,  +(1  +  0. l)x2  <8  (3.3) 

then  xi  and  X2  also  satisfy 

(l  +  x1)x1+(l  +  x2)x2  <8 
Vx;  e[- 0.1,40.1]  i  =  1,2 

Hence,  this  “trick”  can  be  applied  to  all  parameters  and  we  can  get  an 
optimization  problem  which  will  give  us  a  robust  solution.  This  approach  will  also 
define  a  robust  feasible  region  which  is  the  subset  of  the  feasible  region  that  only 
contains  points  feasible  under  worst  case  uncertainty  as  shown  above.  Figure  3.1 
shows  a  comparison  of  the  original  feasible  region  (3.1)  and  the  resulting  robust 
feasible  region  (3.5),  and  the  constraint  functions  of  following  equation  (3.5)  define 
the  robust  feasible  region. 
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min  / (x)  =  -Xj  -  2x2 


s.t. 

gl  =  (1  +  0.  l)Xj  +  (1  +  0.  l)x2  <  8 
g2  =  (-2  +  0.  l)Xj  +  (1  +  0.  l)x2  <  5 
g3  =  (— 1  +  0.  l)Xj  +(-3  +  0.1)x2  <  -10 


(3.5) 
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*1 


Figure  3.1:  Comparison  of  the  Feasible  Region  (Black)  with  the  Robust  Feasible 

Region  (Red)11 

Since  this  is  a  linear  program,  a  solution  will  be  one  of  the  comer  points  of  the 
feasible  region.  The  solution  to  the  deterministic  problem  (3.1)  is  xi  =  1,  X2  =  7,  fix)  = 
-15.  The  solution  to  the  robust  problem  (3.2)  can  be  found  by  looking  at  the  corner 
points  of  the  robust  feasible  region  which  gives  x\  =  1,  X2  =  69/11  (approximately 
6.27),  j{x)  =  -149/11  (approximately  -13.54).  Clearly,  finding  this  robust  feasible 
region  greatly  simplifies  the  robust  optimization  problem.  The  motivation  behind  the 
modified  Benders  method  was  to  find  this  feasible  region  and  then  solve  the  easier 
optimization  problem  (3.5). 


11  Both  the  feasible  region  and  the  robust  feasible  region  are  the  regions  enclosed  by  the  respective 
black  and  red  lines. 
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Recall  that  the  formulation  from  Chapter  2  for  robust  optimization  problems 
(with  objective  robustness  included  in  the  constraints)  is  given  by 
min  fix) 

X 

S.t. 

gj(x,x)<0  j  =  (3.6) 

xeRn,xeRn* 

Vx  €  [-Ax,  Ax] 

The  end  goal  of  this  chapter  is  to  solve  problem  (3.6).  The  method  used  is  a 
modification  of  Benders  decomposition  to  be  described  later.  As  described  in  Section 
2.4.1,  Benders  decomposition  decomposes  an  optimization  problem  into  a  master 
problem  and  a  subproblem,  with  the  variables  being  divided  into  complicating  and 
uncomplicating  ones.  In  (3.6),  x  are  the  uncomplicating  variables  and  x  are  the 
complicating  ones.  As  in  Benders  decomposition,  an  auxiliary  function  of  x  will  be 
defined.  A  set  of  theoretical  results  will  then  be  proven  about  the  complicating 
variables  x  and  the  auxiliary  function.  The  first  set  of  theoretical  results  will  show 
that  an  application  of  standard  Benders  decomposition  to  (3.6)  when  the  objective  and 
constraint  functions  are  linear  will  yield  a  globally  optimal  robust  solution  (Algorithm 
3.1).  Then,  an  assumption  on  the  quasiconvexity  of  the  constraint  functions  gy(x,x) 

will  be  made  to  simplify  the  application  of  a  modified  Benders  decomposition.  This 
modified  Benders  decomposition  with  modified  Benders  cuts  will  then  be  applied  to 
solve  (3.6)  when  the  constraint  functions  gJ (x, x)  are  quasiconvex  (heuristic 
Algorithm  3.2).  Finally,  a  third  heuristic  is  also  presented  which  can  be  used  to  solve 
(3.6)  when  the  constraint  functions  g . (x, x)  are  nonlinear  (not  necessarily 

quasiconvex). 
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To  prove  the  theoretical  results  in  this  chapter,  certain  assumptions  have  to  be 
made  about  optimization  problem  (3.6).  In  Section  2.2.1,  assumptions  on  the 
functions / and  gj  being  continuous  were  stated.  While  the  new  Assumptions  3.1  and 
3.2  may  be  relaxed  for  numerical  application  of  the  algorithms  presented  later,  the 
theoretical  results  depend  on  them.  The  following  are  these  assumptions  and  they 
have  to  do  with  the  existence  of  solutions. 

Assumption  3.1:  The  constraints  gj  in  (3.6),  for  any  fixed  value  of  uncertainty  x , 
form  a  convex,  compact,  nonempty  feasible  region  over  x. 

Assumption  3.2:  A  globally  optimal  robust  solution  to  (3.6)  always  exists. 

Assumption  3.1  ensures  that  continuous  functions  gj,  j  =  I .... ,  J  are  over  a 
nonempty  compact  set  so  they  obtain  their  maximum  within  this  set  via  the 
Weierstrass  Theorem  (Royden,  1988).  The  convexity  of  the  feasible  region  is 
included  to  ensure  that  a  gradient-based  algorithm  can  be  applied  successfully. 
Assumption  3.2  is  stronger,  and  assumes  that  a  solution  exists  to  the  robust 
optimization  problem,  while  Assumption  3.1  does  not  take  into  account  the 
Vx  e  [-Ax,  Ax]  clause  in  (3.6).  Existence  of  solutions  to  robust  optimization  problems 
are  difficult  to  prove.  The  presence  of  uncertainty  means  that  with  large  enough 
values  of  |Ax|,  there  may  not  be  even  a  feasible  solution  to  the  robust  optimization 
problem,  let  alone  a  globally  optimal  robust  solution.  However,  for  example,  problem 
(3.2)  has  an  optimal  robust  solution.  The  algorithms  in  this  chapter  can  be  used  with 
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solvers  which  could  detect  if  an  optimization  problem  did  not  have  a  feasible 
solution. 

The  goal  is  to  obtain  values  of  x  such  that  the  formulation  (3.6)  gives  an 
optimal  solution  to  x  regardless  of  the  values  of  x  .  Since  this  chapter  only  considers 
the  worst-case  analysis,  the  method  aims  to  get  the  “worst”  values  of  x  for  this 
problem  (3.6).  These  are  called  “interval-optimal”  values,  as  defined  next. 

Definition  3.1:  Interval-optimal :  An  interval-optimal  value  for  a  particular 
candidate  solution  xc  and  a  constraint  function  gj  (for  one  j  =  1 , . . . ,  J)  is  defined  as  a 
point  xc  e  [-  Ar,+Ax]  such  that  g.  (xc ,  x)  <  g.  (xc ,  xc )  for  all  realizations  of 

x  e  [- Ar,+Ax].  The  point  xc  e  [-  Ar,+Ar]  is  a  particular  value  of  the  x  such  that  the 
constraint  attains  its  maximum  value  at  that  particular  value  of  uncertainty. 

An  interval-optimal  point  can  be  thought  of  as  the  value  of  uncertainty  x  that 
maximizes  the  value  of  gj  over  all  other  realizations  of  uncertainty  for  a  fixed  value  of 
x.  The  next  definition  takes  this  further. 

Definition  3.2:  Globally  Interval-optimal.  A  globally  interval-optimal  value  for  a 
particular  candidate  solution  xc  and  set  of  constraint  functions  gf,  j  =  1,...,  J;  is 
defined  as  a  point  xc  e  [-  Ar,+Ar]  such  that  g ;  (x' ,  x)  <  max  g ;  (x1 ,  x' )  for  all 

1  j  1 

realizations  of  x  e  [-  Ar,+Ar] .  The  point  xc  e  [-  Ar,+Ar]  is  a  particular  value  of  the 
x  such  that  the  constraints  attain  their  global  maximum  value  at  that  particular  value 
of  uncertainty. 
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Note  that  any  globally  interval-optimal  point  is  interval-optimal  for  at  least 
one  of  the  constraint  functions  gj,  j  =  1 J.  From  example  (3.2),  for  the  constraint 


gi  and  the  candidate  solution 


xi  =  1,  X2  =  69/11  the  associated  interval- 


\x2J 

optimal  value  of  x  is  x;  =0.1,  i  =  1,...,  6.  This  also  happens  to  be  the  associated 

globally  interval-optimal  value  of  x  for  the  solution  xi  =  1,  x2  =  69/1 1  and  the  set  of 
constraints  gi,g2,  g3. 

The  following  lemma  makes  the  connection  between  a  robust  point  and  its 
globally  interval-optimal  point.  In  equation  (3.5),  the  globally  interval-optimal  values 
of  the  uncertainty  elements  helped  detennine  the  robust  solution.  Lemma  3.1  further 
strengthens  this  connection  between  a  robust  point  and  a  globally  interval-optimal 
point. 


Lemma  3.1:  A  candidate  solution  (xc)  for  problem  (3.6)  is  a  robust  point  if  and 
only  if  its  globally  interval-optimal  point 

x  e  [— Ax,+Ax] 


is  such  that 


maxg.(xc,xc)  <  0_ 

j  J 

Proof:  If  (xc)  is  a  robust  point  (Definition  2.4),  then  it  must  be  true  that 
maxg  .(xc,x)<0  for  all  realizations  of  x  e  [-  Ax,+Ax].  Hence,  this  implies  that  for 

the  associated  globally  interval-optimal  point  (xc),  maxgy(xc,xc)  <  0  as 
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xc  e[-Ax,+Ar].  For  the  other  side  of  the  if  and  only  if  argument,  suppose  the 

associated  globally  interval-optimal  point  has  max  g.(xc,xc)  <  0 .  Then  by  the 

i  1 

definition  of  globally  interval-optimal,  max  g  (xc,x)  <  0  for  all  realizations  of 

i  1 

x  e  [-  Ar,+Ar]  which  implies  that  (xc)  is  a  robust  point.* 

The  next  step  is  to  relate  Definition  3.2  to  Benders  decomposition  as 

explained  in  Section  2.4.1.  The  robust  optimization  problem  (3.6)  will  be  solved 

using  a  modification  of  Benders  decomposition.  In  this  modification,  the  uncertainty 

variables  x  will  be  the  complicating  variables.  Since  there  is  a  need  for  an  auxiliary 

12 

function  as  in  equation  (2.11),  define  the  following  function 
«„(*)  =  min  f(x) 

X 

s.t. 

gj(x,x)<  0  j  =  (3.7) 

x  e  R" 

where  x  e  R"u 

Before  proceeding,  it  is  important  to  define  one  more  term  and  make  an 
assumption.  The  next  definition  is  a  slightly  different  one  than  Definitions  3.1  and 
3.2,  but  is  related  to  our  motivation  for  finding  interval-optimal  points  using  a 
modified  Benders  decomposition. 


12  By  Assumption  3.2,  a  solution  to  (3.7)  always  exists.  This  is  because  the  optimization  problem  (3.7) 
is  a  relaxed  version  of  the  optimization  problem  (3.6). 
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Definition  3.3:  Worst-Case  Uncertainty.  A  worst-case  uncertainty  value 
x"’c  e  [-  Ax,+Ax]  for  optimization  problem  (3.6)  is  such  that  when  x  =  x"'is  fixed  in 
(3.6),  the  solution  of  (3.8)  below  yields  a  globally  optimal-robust  solution, 
min  /(x,x) 

X 

S.t. 

gj(x,x)<  0  j  =  (3.8) 

xeRn,xeR"“ 

X  =  xwc 

Note  that  a  worst-case  uncertainty  value  differs  from  an  interval-optimal  or 
globally  interval-optimal  value  of  uncertainty  in  that  a  worst-case  uncertainty  value 
does  not  have  an  associated  predetermined  variable  x  (but  it  is  associated  with  a 
globally  optimal  robust  solution  after  solving  (3.8))  and  is  for  the  entire  optimization 
problem.  But  it  is  trivial  to  note  that  a  worst-case  uncertainty  value  of  x  is  a  globally 
interval-optimal  point  of  uncertainty  for  some  globally  optimal  robust  solution.  The 
next  assumption  is  required  for  the  theoretical  background  of  the  modified  Benders 
decomposition  presented  in  this  chapter. 

Assumption  3.3:  A  worst-case  uncertainty  value  exists  for  robust  optimization 
problem  (3.6)  and  is  a  globally  interval-optimal  point  for  a  globally  optimal  robust 
solution  x*. 

Note  that  Assumption  3.3  ensures  that  finding  worst-case  uncertainty  values 
enables  us  to  find  a  globally  optimal  robust  solution.  Problem  (3.2)  was  an  example 
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of  a  robust  optimization  problem  where  x,.  =  0.1,  i  =  1,...,  6  was  figured  out  to  be  the 
worst-case  uncertainty  value. 

As  an  example,  linear  programs  satisfy  Assumption  3.3.  In  general,  a  linear 

nx  nu 

constraint  function  g,  can  be  written  as  gj(x,x )  =  ^  cixi  +  ^  dixi  +  C  where  C  is  a 

i=l  i= 1 

real  number.  Here,  as  in  example  (3.2),  if  there  is  a  variable  x  such  that 

nx  nu 

sM)  =  YJcixi  +  2J  dt  |  Axz-  +  C  <  0 ,  then  it  is  also  true  that 

i=i  i=i 

nx  nu 

g  (x,  x)  =  ^ cixi  +  ^  t/,x,.  +  C  <  0  for  all  x  e  [-  Ax,+Ax].  Therefore,  the  interval- 

i= 1  /=1 

optimal  value  of  x,  can  be  calculated  to  be  +  Ax,  if  t/,  is  positive  and  -  Ax,  if  dj  is 
negative.  From  this  argument,  for  all  the  gj  constraints  a  globally  interval-optimal 
value  can  be  calculated  and  so  can  a  worst-case  uncertainty  value.  For  more 
information  on  problems  that  satisfy  Assumption  3.3,  please  refer  to  (Ben-Tal  et  ah, 
2009). 

The  following  lemma  shows  a  property  of  the  new  auxiliary  function  (3.7) 
which  connects  a  globally  optimal  robust  point  to  its  globally  interval-optimal  point. 
This  will  later  be  used  in  modifying  Benders  decomposition  to  obtain  solutions  to 
robust  optimization  problems. 

Recall  that  for  problem  (3.2),  the  globally  optimal  robust  point  was  xi  =  1,  X2 
=  69/11  and  its  associated  globally  interval-optimal  point  (and  the  worst-case 
uncertainty  value)  was  x,.  =0.1,  i  =  1,...,  6.  Note  that  for  example  (3.2), 

a„(0.1)  = -149/11 ,  which  happens  to  be  the  function  value  of  the  globally  optimal 
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robust  solution  to  example  (3.2).  This  is  no  coincidence  as  shown  by  the  following 
lemma. 

Lemma  3.2:  Under  Assumptions  3. 1-3.3,  let  x*  be  a  globally  optimal  robust 
solution  (Definition  2.6)  to  (3.6)  and  x  *  an  associated  globally  interval-optimal 
point.  If  (i)  au  (x*)  >  au  (x)  for  all  realizations  of  x  e  [-  Ar,+Ar]  ,  then 
(ii)  au  (x*)  =  /(x*). 

Proof:  Since  x*  is  a  globally  optimal  robust  point,  it  is  automatically  a  robust  point 
(Definition  2.6)  so  by  Lemma  3.1,  maxg  (x*,x)<0  for  all  realizations  of 

i  1 

xe[-Ax,+Ar].  Note  that  the  value  ajx*)  was  calculated  by  minimizing  j{x)  while 
fixing  x  =  x*  in  (3.7).  Since  x*  is  in  the  feasible  region  for  (3.7)  and  by  Assumption 
3.2,  a  solution  always  exists  to  (3.7),  au(x*)  <  /(x*)  .  The  next  step  will  show  with 
the  help  of  a  contradiction  argument  that  au  (x*)  >  / (x*)  . 

Suppose  that  au  (x*)  <  / (x*) .  By  the  statement  of  this  lemma, 
au (x*)  >  au (x)  for  all  x  e  [- Ar,+Ar].  Using  (3.7)  by  fixing  x  =  x*,  let  x' 
(dependent  on  x*)  be  a  solution  to  the  minimization  problem  in  (3.7)  such  that 
au(x*)  =  /(x') .  Then  (i)  implies  f(x')>au(x)  for  all  xe[-Ar,+Ar].  By  our 
contradictory  assumption,  this  also  implies  f(x*)>au(x*)  =  f(x')>au(x)  which 
simplifies  to  fix*)  >  au(x)  for  all  xe[-Ax,+Ax].  Note  that  the  condition 
f(x*)>all(x )  for  all  xe[-Ax,+Ax]  violates  Assumption  3.3.  By  Assumption  3.3, 
there  exists  a  worst-case  uncertainty  value,  i.e.,  there  exists  a  x"’c  e[-Ar,+Ar]  such 
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that  f(x*)  =  au(xwc) .  But  this  would  imply  au(xwc)  >  au(x)  for  all  x  e  [- Ar,+Ar] 
which  is  a  contradiction.  Hence,  this  contradiction  shows  that  au(x*)>  fix*)  ■ 
Combining  the  two  inequalities  au(x*)>  fix*)  and  afx*)  <  fix*)  gives 
au(x*)  =  f(x*)M 

The  next  two  theorems  fonn  the  basis  of  the  modified  Benders  method  to  be 
introduced  later  in  this  chapter.  The  first  shows  a  particular  characteristic  of  a  worst- 
case  value  of  uncertainty.  The  second  shows  that  a  particular  characteristic  of  an 
uncertainty  variable  value  can  be  used  to  find  a  globally  optimal  robust  solution.  The 
modified  Benders  method  of  this  chapter  will  aim  to  find  this  value. 

Theorem  3.1:  Under  Assumptions  3. 1-3.3,  let  the  worst-case  value  of  uncertainty 
for  (3.6)  be  xwc .  Then,  au(xwc)  >  a u (x)  for  all  realizations  of  ice  [-  Ax,+Ax]. 

Proof:  Let  x*  be  a  globally  optimal  robust  solution  to  (3.6).  Then,  by  Definition  3.3, 
afx"')  =  f(x*).  Problem  (3.6)  has  the  same  objective  function  as  (3.7)  but  the 
feasible  region  of  (3.6)  is  a  subset  of  the  feasible  region  of  (3.7).  Therefore,  for  any 
fixed  x  fix*)>  afx) .  Therefore,  au(x"r)  >  au(x)  for  all  realizations  of 
xe[-  Ar,+Ar  J  ■ 

Theorem  3.2:  Under  Assumptions  3. 1-3.3,  suppose  there  exists  a  unique 
uncertainty  value  xc  for  which  auixc)>  afx)  for  all  realizations  of 
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x  <e  [- Ax,+Ax].  Then,  a  solution  to  optimization  problem  (3.9)  will  be  a  globally 
optimal  robust  solution  to  problem  (3.6). 

min  f(x) 

X 

S.t. 

gj(x,x)<  0  j  =  (3.9) 

/V  *  C 

X  =  X 

x  e  R" ,x  e  R'1" 

Proof:  Let  xc  be  a  solution  to  (3.9).  Note  that  au(xc)  =  f{xc )  by  (3.7).  By 
Assumption  3.3,  there  exists  a  worst-case  uncertainty  value  xwc  such  that 
a„(xwc)  =  f(x *)>  where  x*  is  a  globally  optimal  robust  solution.  Since  x*  is  a 
solution  to  (3.6),  it  is  also  feasible  to  (3.9)  as  the  feasible  region  for  (3.6)  is  a  subset 
of  the  feasible  region  for  (3.9).  Hence,  f(xc)<f(x*),  which  implies 
au  (x  )  <  a(x"c)  .  But  according  to  the  statement  of  this  theorem,  au  (x  )  >  au(x)  for 
all  realizations  of  x  e  [-  Ax,+Ax] .  Hence,  xc  =  xwc  because  this  theorem  also  states 

that  this  value  of  xc  is  unique.  Therefore,  xc  =  xwc .  By  Definition  3.3,  (3.9)  gives  a 
globally  optimal  robust  solution.  ■ 

The  purpose  of  Theorem  3.2  is  that  if  the  following  optimization  problem  13 
(3.10)  has  a  unique  solution,  that  solution  can  be  used  to  find  the  solution  to  (3.6). 


13  Note  that  the  function  (Xu  (x)  is  not  known  in  closed  form  but  will  be  later  be  approximated  using  a 
variation  on  Benders  cuts. 
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max  au  (x) 


;  r  .  (3.io) 

ie[-  Ax,  Ax\ 

*  d  n„ 

x  e  R  11 

Theorem  3.2  also  shows  that  finding  globally  interval-optimal  points  can  help 
us  obtain  a  globally  optimal  robust  solution.  For  quasiconvex  constraint  functions,  we 
know  that  the  globally  interval-optimal  values  will  lie  on  one  of  the  endpoints  of  the 
vector  interval14  [-Ax, Ax],  which  follows  directly  from  the  definition  of 
quasiconvexity  provided  in  Chapter  2  and  is  taken  advantage  of  in  the  following 
Corollary  3.1.  Note  that  there  are  2""  such  endpoints,  where  nu  is  the  dimension  of  the 
vector  Ax  and  also  the  dimension  of  the  endpoints.  For  purposes  of  notation,  let  the 
endpoints  of  the  vector  interval  be  denoted  by  (V)i,  ( Kk ,  (Eb,  ...,  (F),„u .  Each 

endpoint  vector  (V)k  ,  k  =  1,...,2"“  is  defined  such  that  each  of  its  elements  (Vj)k  is 
either  Ax,  or  -Ax,-,  i.e.,  {y\  e  {- Ax,- ,  Ax,- }  for  i  =  1,...,  nu.  The  idea  that  the 

maximum  of  the  constraint  functions  lies  on  one  of  the  endpoints  of  the  vector 
interval  can  be  used  to  ascertain  that  any  globally  interval-optimal  point  (and  thus, 
worst-case  uncertainty  value)  will  also  he  on  one  of  the  endpoints  of  the  vector 
interval. 

Corollary  3,1:  If  the  constraint  functions  gj  ,j  =  1,,,.,  J  are  quasiconvex  in  (3.6), 
then  solving  (3.11)  is  equivalent  to  solving  (3.6)  and  solving  (3.12)  is  equivalent  to 


14  This  vector  interval  [—  Ax,  Ax]  is  of  nu  dimensions.  So  the  endpoints  of  this  vector  interval  are 
actually  all  the  corner  points  of  an  n, ,-dimensional  rectangle. 
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solving  (3.7).  Additionally,  the  maximum  value  of  au  (x)  is  achieved  at  one  of  the 


endpoints  {(f), 

min  fix) 

X 

S.t. 

gj(x,x)<  0  j  =  (3.11) 

n  n  *  _  tv  n,. 

IGA  ,IGi(“ 

Vis  {{V )£, 
a„(x)  =  min  f(x) 

X 

S.t. 

gj(x,x)<  0  j  =  (3.12) 

x  e  R" 

where  xe  {(f), 

Proof:  The  aim  of  this  theorem  is  to  show  that  if  gj,  for  ally  =  1,...,  Jis  quasiconvex 
in  x ,  then  Vx  e  {(F ), }{=|  can  replace  the  condition  Vx  e  [-Ax,  Ax]  in  (3.6).  Note  that 
gj  (x,  xt )  =  gj  (x,  A(-Axi )  +  (1  -  A(Axj )))  <  max  {gy.  (x-Ax, ),  g.  (x,  Ax,. )}  for  all 

/le[0,l]  and  a  real-valued  x,.  e  [-Ax,., Ax,].  For  any  x,  if 

gj  (x,  x)  <  0 ,  Vx  e  [-Ax,  Ax] ,  then  g .  (x,  x)  <  0 ,  Vx  e  {(F ),  }{=]  because 

{(F ),  ]2k= \  [-Ax,  Ax] .  Moreover,  for  any  x  if  g  j  (x,  x)  <  0 ,  Vx  e  {(F ),  }2k=x ,  then 

gj  (x,  x)  <  0 ,  Vx  e  [-Ax,  Ax]  because  gj  is  quasiconvex  in  x.  Hence, 

gj  (x,  x)  <  0 ,  Vx  e  {(F ),  ,  if  and  only  if  g  /  (x,  x)  <  0 ,  Vx  e  [-Ax,  Ax] .  Therefore, 

:1  can  replace  the  condition  Vx  e  [-Ax,  Ax]  in  (3.6)  whenever  gj,  for  all  j 
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=  1,...,  Jis  quasiconvex  in  x  .  Therefore,  solving  (3.11)  is  the  same  as  solving  (3.6) 
and  solving  (3.7)  is  the  same  as  solving  (3.12). 

Note  that  a  value  of  x  that  maximizes  the  constraint  gj  for  a  fixed  value  of  x 
in  the  expression  max  {g.(x,x)}  is  an  interval-optimal  value.  Moreover,  a  value  of 

x  that  maximizes  the  expression  max|  max  {g/(x,x)}j  is  a  globally  interval- 

optimal  value  of  x .  Hence,  the  globally  interval-optimal  values  of  x  lie  on  the 
endpoints {(f)* }^=”  .  In  particular,  by  Assumption  3.3,  a  worst-case  value  of 
uncertainty  xwc  for  problem  (3.11)  exists  and  is  a  globally  interval-optimal  value. 
Hence,  xwc  also  lies  on  one  of  the  endpoints.  By  Theorem  3.1,  au(x"c)  >  au(x)  for 
all  realizations  of  x  e  [-  Ax,+Ax],  which  is  the  maximum  value  of  ccjx) .  Hence,  the 
maximum  value  of  au  (x)  is  achieved  at  one  of  the  endpoints.* 

The  next  section  provides  a  modified  Benders  decomposition  method  to  solve 
robust  linear  programs.  Note  that  so  far  we  have  proven  facts  about  the  auxiliary 
function  au  (x) .  Standard  Benders  decomposition  approximates  this  function  using 

standard  Benders  cuts.  With  the  support  of  theoretical  results,  the  first  algorithm  in 
the  next  chapter  will  apply  standard  Benders  decomposition  to  obtain  a  solution  to 
(3.6).  Two  further  heuristic  algorithms  are  provided  to  solve  (3.6)  when  the  constraint 
functions  are  quasiconvex  and  nonlinear,  respectively. 
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3.3  Modified  Benders  Decomposition 


3.3.1.  Formulation  of  Approach:  Solving  Robust  Linear  Programs 

The  strategy  will  be  to  find  optimal  values  for  variables  x  and  interval-optimal  values 
for  x .  One  can  think  of  this  as  attempting  to  check  the  robustness  of  a  candidate 
solution  by  partitioning  the  uncertainty  interval  and  checking  feasibility  at  each  point. 
Clearly,  if  all  constraints  are  feasible  when  fixed  with  the  interval-optimal  values  for 
uncertainty  elements,  then  the  candidate  solution  is  robust. 

Standard  Benders  decomposition  is  advantageous  when  the  problem  structure 
dictates  that  fixing  certain  variables  will  lead  to  a  simpler  problem  to  solve.  The  idea 
behind  Benders  decomposition  is  to  fix  a  set  of  complicating  variables  and  solve  a 
resulting  simpler  subproblem  while  iterating  between  it  and  solving  a  master  problem 
that  computes  values  for  the  complicating  variables.  The  robust  optimization  problem 
(3.6)  also  has  a  simple  structure  when  certain  variables  are  fixed.  Fixing  x  results  in 
an  optimization  problem  much  simpler  to  solve  than  a  robust  optimization  problem. 

The  Benders  cuts  added  in  (2.14)  serve  to  approximate  the  function  a 
described  in  (2.11).  Since  the  objective  function  in  (3.7)  is  being  maximized  by 
equation  (3.10),  for  the  successful  application  of  Benders  decomposition,  the  function 
au  (x)  needs  to  be  concave. 

Theorem  3.3:  For  linear  objective  and  constraint  functions  in  (3.6),  the  function 
au  (x)  is  concave. 
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Proof:  This  is  a  well-known  result  from  linear  programming  theory  (Murty,  1983). 


The  function  au  (x)  is  piecewise-linear,  continuous,  and  concave.  ■ 


Because  Theorem  3.2  requires  a  unique  maximum  value  of  x  to  guarantee  a 
solution,  the  following  lemma  gives  conditions  under  which  this  is  possible  for  linear 
robust  optimization  problems.  Before  proceeding,  the  definition  of  a  slope  between 
two  points  is  needed. 


Definition  3.4:  Slope:  The  slope  of  the  functionary)  :  R"“  — »  R  between  two  vectors 


y1  e  R"‘ ,  y2  e  R"’  is  defined  as  the  vector  A  e  Rn"  where  At  =  1 ^  for  each 


yf-y) 


element  i=  1,...,  nu. 


Lemma  3.3:  Let  au(x)  be  a  piecewise  linear,  continuous,  concave  function  over 
[-Ax,  Ax].  Then  au (x)  achieves  its  maximum  at  a  point  xwc  e [-Ax, Ax] .  Now 

suppose  this  maximum  is  achieved  at  one  of  the  endpoint  vectors  {(V )jt  fkl\  and 
suppose  the  slope  between  xwc  and  any  other  vector  x  e  [-Ax,  Ax]  is  nonzero 
(every  element  of  the  slope  between  the  two  points  is  nonzero).  Then  au  (x)  has  a 
unique  maximum. 

Proof:  Since  au  (x)  is  a  continuous  function  over  a  nonempty  compact  set,  the  vector 
interval  [-Ax, Ax],  by  the  Weierstrass  theorem  (Royden,  1988)  ajx)  achieves  its 
maximum  at  a  vector  in  [-Ax,  Ax] . 
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Let  the  maximum  of  au  (x)  be  achieved  at  xwc  e  \(V )k  ,  which  is  one  of  the 
endpoints  .  We  want  to  show  that  this  point  is  unique.  This  proof  will  follow  a 
contradiction  argument.  Suppose  there  exists  another  distinct  point  xVfc  such  that 
au(xwc)  =  au(x')  and  au (x' )  >  au (x)  for  all  x  e  [-Ax,  Ax] .  Since  au (x)  has  nonzero 
slope  between  xwc  and  any  other  point  in  [-Ax,  Ax] ,  there  exists  a  point  xm  such  that 
xm  *x',xm  ^  xwc  which  is  a  strict  convex  combination  of  xwc  and  x'  such  that 
au(xm)  ^  au (x"c) .  Hence,  there  exists  a  2  e  (0,1 )  such  that  xm  =  Axwc  +  (1  - 2)x' . 
Since  au(x)  is  concave,  a(xm)  >  Aa(xwc)  +  (\-A)a(x')  which  implies 
a(xm)>a(xwc)  because  au(x"c)  =  aN(x' ) .  But  we  had  assumed  that 
au(xm)  au(x"’c)  so  it  must  be  that  a(xn)  >  a(x"c) .  This  is  a  contradiction  as  it 
violates  the  statement  in  the  theorem  that  a  maximum  is  achieved  at  one  of  the 
endpoints.  Therefore,  the  assumption  au(xwc)  =  au(x')  ends  up  concluding  a 
contradictory  result  and  au  (x)  has  a  unique  maximum.  ■ 

The  use  of  xwc  as  the  notation  for  the  point  where  an  (x)  achieves  its 
maximum  was  not  coincidental.  It  is  used  to  relate  the  result  of  Lemma  3.3  to 
Theorem  3.1  and  Corollary  3.1.  Note  that  there  exist  other  functions  than  linear  that 
are  both  concave  and  achieve  their  maximum  at  an  endpoint  of  their  interval  domain. 
Examples  are  log(y)  over  [1,100],  -y  over  [0,1],  etc.  The  following  is  Algorithm  3.1 
for  solving  robust  optimization  problems  with  linear  objective  and  constraint 
functions. 
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Algorithm  3.1  (Standard  Benders  Method): 


Step  0:  Set  iteration  counter  (it)  to  0.  Pick  a  small  positive  constant  for  tolerance  ( tol ). 


Step  1:  Set  iteration  counter  (it)  to  it  =  it  +  1.  The  original  master  problem  will  be: 


max  au 

a„,x 


S.t. 

-  Ax  <  x  <  Ax 


a  <  a 

w 'u 


(3.13) 


The  bounds  on  au  are  user-defined15  depending  on  the  problem.  Solving  the  above 


problem  gives  au  =  a*  and  x  =  x%ed  . 


Step  2:  Fix  the  values  of  the  complicating  variables  x ,  and  then  solve  the  following 
subproblem  as  in  the  standard  Benders  decomposition  method. 

w  =  min  /  (x) 


s.t. 


gj(x,x)  <  0  j  = 
where 


(3.14) 


x  =  x 


fixed 


(A"  dual) 


15  Further  suggestions  are  available  to  achieve  this  upper  bound  in  (Conejo  et  at,  2006).  For  Algorithm 
3.1,  a  good  value  of  the  upper  bound  can  be  achieved  by  maximizing  (as  opposed  to  minimizing)  the 
objective  function  in  (3.6)  over  the  entire  space  of  decision  and  uncertainty  variables.  This  is  a  definite 
upper  bound  to  au  as  (Xu  is  an  auxiliary  function  to  (3.6),  given  Assumption  (3.2)  holds. 
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Step  3 :  Check  for  convergence.  Set  zsub  =  w  and  zmas  =  a* .  If  the  difference 

(| Zsub  -  Zmas I)/  Zsub  ^  to1  then  St°P' 


Step  4:  Add  a  Benders  cut  to  the  master  problem  (3.13). 


Step  1  (returned):  Solve  the  following  master  problem  after  adding  the  Benders  cut 


max  a 

a,„x 

S.t. 

-  Ax  <  x  <  Ax 


((x)r-(xf0/)r) 


au<cc 


max 

u 


(3.15) 


Return  to  Step  2  and  proceed  in  this  manner  until  convergence  is  met. 

Theorem  3.4:  If  at  the  final  iteration  X‘sol  ^  0  for  every  element  of  Kon 
Algorithm  3.1  converges  to  a  globally  optimal  robust  solution  x*  of  (3.6)  and 
worst-case  uncertainty  value  x  *  in  a  finite  number  of  steps. 

Proof:  By  the  theory  of  Benders  decomposition  for  linear  programs  (Benders,  1962), 
Algorithm  3.1  converges  to  a  maximum  value  for  au  in  a  finite  number  of  steps.  The 

algorithm  also  provides  x*  and  x*  such  that  au  (x*)  =  /(x*) .  If  at  the  final  iteration 
Kc  0  f°r  each  element,  then  the  function  au  approximated  by  the  Benders  cuts 
does  not  have  zero  slope  between  x  *  and  any  other  point  in  [-Ax,  Ax] .  Moreover,  by 
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Lemma  3.3,  au  has  a  unique  maximum  point  at  x*  .  By  Theorem  3.2,  x*  is  a  worst- 
case  uncertainty  value,  and  fixing  that  value  in  (3.9)  gives  a  globally  optimal  robust 
solution  to  (3.6).« 

Note  that  if  at  the  final  iteration  a  cut  is  added  where  A°°l  =  0 ,  then  Algorithm 
3.1  need  not  necessarily  converge  to  a  globally  optimal  robust  solution  because 
Theorem  3.2  requires  this  maximum  point  to  be  unique.  If  X°t'  =  0  at  the  final 
iteration,  then  there  can  be  several  points  that  maximize  au ,  not  all  necessarily  a 

worst-case  uncertainty  value.  Algorithm  3.1  could  then  converge  to  a  point  that  was 
not  a  robust  point.  In  that  case,  the  use  of  heuristic  Algorithm  3.3  is  needed. 

3.3.2.  Formulation  of  Approach:  Solving  Robust  Optimization  Problems 

with  Quasiconvex  Constraints 

For  convergence  of  the  Benders  decomposition  algorithm,  the  function  au  needs  to  be 
concave.  There  is  a  larger  class  of  functions  than  simply  linear  programs  for  which 
these  conditions  are  valid.  Indeed,  for  many  engineering  applications  as  well  as 
numerical  examples,  local  concavity  of  au  can  be  sufficient  (Conejo  et  ah,  2006). 

However,  due  to  the  worst-case  analysis  performed,  au  is  quasiconvex  as 
given  by  Corollary  3.1.  Unfortunately  standard  Benders  cuts  cannot  be  used  to 
approximate  quasiconvex  functions  (Conejo  et  ah,  2006).  The  reason  is  that  optimal 
solutions  may  be  omitted  when  cuts  are  added.  Our  advantage  in  a  robust 
optimization  setting  with  quasiconvex  constraints  as  in  problem  (3.1 1)  is  that  we  only 
need  good  approximations  to  the  functions  at  the  endpoints.  Approximations  of  the 
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function  au  are  not  needed  in  between  the  endpoints,  as  the  function  attains  its 
maximum  at  the  endpoints.  Figure  3.2  shows  the  idea  behind  these  new  cuts.  The 
very  top  horizontal  cut  (labeled  as  “Cut  0”)  is  an  upper  bound  set  for  au  as  would 
normally  occur  in  Benders  decomposition.  The  numbers  next  to  the  cuts  show  the 
order  of  the  cuts  made  in  the  iterative  process.  At  iteration  it  a  new  modified  Benders 
cut  added  to  the  master  problem  looks  like  the  following. 


sol  * sol 

i,  ,Xu 


f(xsolX)-f(xSO,X\)  ,  T 

)  + JK  •  ,T  ,tx  ((x)r 


/  *sol\T  \ 

<xit  )  ) 


(3.16) 


This  cut  is  one  way  to  approximate  the  function  around  the  endpoints  and  see  which 
value  of  au  at  the  endpoints  is  larger  as  shown  in  Figure  3.2. 


16  The  dashed  line  denotes  the  quasiconvex  function  Clu  that  is  supposed  to  be  approximated. 
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The  following  Algorithm  3.2  describes  the  method  for  quasiconvex  constraints. 
Algorithm  3.2  differs  from  Algorithm  3.1  in  that  the  modified  Benders  cuts  described 
above  are  used. 

Algorithm  3.2  (Heuristic  Algorithm  for  Robust  Optimization  Problems  with 
Quasiconvex  Constraint  Functions): 

Step  0:  Set  iteration  counter  (it)  to  0.  Pick  a  small  positive  constant  for  tolerance  ( tol ). 

Step  1:  Set  iteration  counter  (it)  to  it  =  it  +  1.  The  original  master  problem  will  be: 

max  au 

au  ,x 

S.t. 

-Ax<x<Ax  (3-17) 

a  <  a™x 

Starting  point  au  =  , x  =  -At 

The  bounds  on  au  are  user-defined  depending  on  the  problem.  A  good  value  is  an 
optimal  objective  function  value  for  the  non-robust  nominal  problem.  Solving  the 
above  problem  gives  au  =  a"  and  x  =  xltflxed  . 

Step  2:  Fix  the  values  of  the  complicating  variables  x ,  and  then  solve  the  following 
subproblem  as  in  the  standard  Benders  decomposition  method. 
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w  =  min  fx(x) 


X 

S.t. 

gj(x,x)<  0  j  =  (3.18) 

where 

Y  —  Yl‘ 

A  A  fixed 


Step  3 :  Check  for  convergence.  Set  zsub  =  w  and  zmas  =  a"  .  If  the  difference 

(j Zsub  -  Zma^!  Zsub  ^  to1  then  Stop. 

Step  4:  Add  a  modified  Benders  cut  to  the  master  problem.  If  this  is  any  iteration 
greater  than  one,  do  not  add  an  additional  cut  but  just  update  the  previous  cut.  To 
problem  (3.17),  add  the  modified  Benders  cut  (3.16). 


Step  1  (returned):  Solve  the  following  master  problem  after  adding  the  modified 
Benders  cuts17 


max  au 

au,x 

S.t. 


Ax<x<  Ax 


au<f(xf,rl 


(3.19) 


)  + 


fix 


sol 


sol 
5  A it 


)-f{xs°'X\) 


sol' 
5  ^  it- 


(xfY-iZZ i) 


*  sol  \T 


((x)T-(xf)T) 


a„  <  a 


max 

u 


Return  to  Step  2  and  proceed  in  this  manner  until  convergence  is  met. 


17  Note,  for  the  first  iteration  take  X-t-\  eclual  to  the  value  of  At  . 
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3.3.3.  Formulation  of  Approach:  Solving  Robust  Optimization  Problems 
with  Nonlinear  Constraints 

To  extend  the  modified  Benders  decomposition  method  to  general  nonlinear 
constraints,  the  strategy  will  be  to  partition  uncertainty  intervals  for  uncertainty 
variables  and  attempt  to  find  the  interval-optimal  points.  Essentially,  one  can  think  of 
this  as  attempting  to  check  the  robustness  of  a  candidate  solution  by  partitioning  the 
uncertainty  interval  by  points  and  checking  feasibility  at  each  point.  Clearly,  if  all 
constraints  are  feasible  when  fixed  with  the  interval-optimal  values  for  uncertainty 
elements,  then  the  candidate  solution  is  robust.  Partitions  can  be  selected  depending 
on  which  type  of  constraint  functions  have  uncertainty.  In  particular,  quasiconvex 
constraint  functions  are  simplest  to  consider. 

Hence,  under  uncertainty  in  x,  the  maximum  value  of  the  function  g  lies  on 
one  of  the  endpoints  of  uncertainty  when  g  is  quasiconvex.  So  finding  the  interval- 
optimal  point  for  quasiconvex  constraint  functions  under  uncertainty  entails  checking 
one  of  the  two  endpoints  of  uncertainty. 

However,  for  nonlinear,  not  necessarily  quasiconvex  constraint  functions,  the 
interval-optimal  values  need  not  lie  on  the  endpoints.  For  this,  the  constraint  functions 
in  the  uncertainty  interval  range  need  to  be  checked  at  intermediary  points  to  find  the 
interval-optimal  points.  Figure  3.3  shows  how  checking  further  points  helps.  For 
quasiconvex  constraint  functions,  Figure  3.3(a),  only  the  endpoints  need  to  be 
checked.  For  general  nonlinear  constraints,  however,  problems  might  be  encountered 
if  enough  points  within  the  uncertainty  interval  are  not  checked.  This  is  shown  in 
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Figure  3.3(b)  where  the  method  fails  if  enough  points  are  not  selected.  Since 

quadratic  constraints  are  symmetric,  for  concave  quadratic  constraints  in  particular 

(that  are  not  quasiconvex)  checking  three  points  (endpoints  plus  central  point  of 

uncertainty  interval,  where  x  =  0  )  is  enough  to  guarantee  a  robust  solution  (in  general 

true  for  all  symmetric  concave  constraints),  Figure  3.3(c).  Selecting  more  points, 

Figure  3.3(d),  solves  this  problem.  Points  maybe  selected  according  to  the  accuracy 

desired  for  a  locally  optimal  robust  solution. 

18. 

Checking  additional  points  entails  adding  additional  constraints  that  have 
different  uncertainty  variables  xk  e  [-  Axk ,  Axk  ] ,  k  =  1  with  uncertainty  ranges 

that  are  subsets  of  the  original  uncertainty  ranges,  i.e.  [-  Axk ,  Axk  ]  cz  [-  Ax,  Ax]  for  all 
k  =  1  In  particular,  the  modified  Benders  method  considers  a  uniform 

distribution  of  these  points  to  be  checked  with  Axk  =  A x/  k  ,  k  =  1,. .  ,,K.  To  check  for 

center  points,  the  constraints  with  g;.(x,0)  <  0,  j  =  1,..,  J  need  to  be  added.  Note  that 

this  constraint  is  just  the  constraint  gj  without  any  consideration  for  uncertainty.  Since 
0  is  directly  in  the  middle  of  the  uncertainty  interval,  the  constraint 
gj (x,0)  <  0,  j  =  1,..,  J  is  simply  the  constraint  with  no  uncertainty. 


18  Usually  the  number  of  additional  points  to  be  checked  depends  on  the  type  of  constraint  functions  as 
well  as  the  accuracy  desired.  A  good  baseline  for  nonlinear  constraint  functions  is  to  uniformly 
partition  the  uncertainty  interval  into  enough  additional  points  so  that  the  distance  between  any  two 
points  is  less  than  or  equal  to  a  preset  tolerance  for  the  problem.  More  points  signify  more  accuracy. 
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Figure  3.3:  Checking  Feasibility  by  Interval-Optimal  Points  for  Constraints:  (a) 
Quasiconvex:  Successful  Check  by  Endpoints;  (b)  Non-convex:  Failed  Check  due 
to  Insufficient  Number  of  Points;  (c)  Symmetric  Concave:  Successful  Check  by 
Middle  Point  and  Endpoints;  (d)  Non-convex:  Successful  Check  by  Sufficient 

Number  of  Intermediary  Points 

So,  the  formulation  changes  from  (3.11)  by  adding  further  sample  points  (2 K 
sample  points). 


min  f{x) 

X 

S.t. 

gj(x,x)<  0  j  = 
g7(x,0)<0 

gj(x,xk)<  0  k  =  l,...,K,j  = 
xeR"  ,xeR 
V.r  e  {-  Ax,  Ar}"" 

\/xk  e  {-  Axk ,  Axk k=l,...,K 
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Here,  Axk ,  k  =  l,...,  K  are  the  different  sample  points  for  the  uncertainty  interval  as 


described  by  the  black  dots  in  Figure  3.3.  Note  that  0<Axa.  <Ax.  The  uncertainty 

elements  (with  a  superscript  A)  only  take  on  two  values  (per  element  of  vector)  each. 
Since  all  the  extra  constraints  from  (3.20)  when  compared  to  (3.11)  can  be 
incorporated  into  the  constraints  in  (3.11),  from  now  on  the  rest  of  this  chapter  will 
assume  that  (3.11)  has  had  a  thorough  sample  of  points  such  that  within  each  interval 
of  uncertainty,  the  constraint  functions  are  quasiconvex. 

To  speed  up  computation  of  the  proposed  approach,  gradient-based 
optimization  algorithms  (Bazaraa  et  ah,  1993)  are  used  as  opposed  to  population- 
based  optimization  ones  (such  as  Genetic  Algorithms  or  Simulated  Annealing) 
(Davis,  1987).  In  particular,  the  nonlinear  solvers  CONOPT  in  GAMS  (GAMS,  2010) 
which  are  gradient-based  were  used  for  all  test  problems  except  the  last  one  on  heat 
exchanger  design.  Since  the  code  for  the  heat  exchanger  design  problem  was  already 
available  and  coded  in  MATLAB,  fmincon  (MATLAB,  2008)  was  used  for  that 
particular  problem. 

The  following  describes  the  modified  Benders  decomposition  algorithm  where 
the  constraint  functions  are  quasiconvex  within  each  interval  of  uncertainty. 

Algorithm  3.3  (Heuristic  Algorithm  for  Robust  Optimization  Problems  with 
Nonlinear  Constraint  Functions): 

Step  0:  Proceed  exactly  as  Algorithm  3.2  for  problem  (3.20) 
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f(xo1  x°')-  f(xso1  X°! ) 

Step  1:  If  at  any  iteration,  : - "  ’  - "  -  ’  "  1  =  0 ,  add  more19  sample  points  in 


wr  -(*&) 


(3.20)  by  making  a  uniform  partition.  Return  to  Step  0. 


Note  that,  theoretically,  convergence  is  not  guaranteed  in  Algorithm  3.3.  But 
the  numerical  results  suggest  that  Algorithm  3.3  is  applicable  to  a  wide  variety  of 
problems.  Now  some  computational  costs  based  on  numerical  evidence  will  be 
provided.  Consider  a  single-objective,  robust  optimization  problem  with  V  variables, 
J  constraints,  P  parameters,  and  N  uncertainty  variables.  One  function  calk  is  defined 
as  any  instance  where  the  solver  calls  an  objective  function,  constraint,  or  other  value 
or  assignment  in  the  optimization  problem.  Table  3.1  gives  the  maximum  number  of 
function  calls  possible  through  one  iteration  of  the  modified  Benders  method.  This 
analysis  is  only  based  on  numerical  evidence  that  the  method  finds  a  locally  optimal 
robust  solution  through  Algorithm  3.3. 


19  In  practice,  double  the  number  of  points  as  was  done  for  the  numerical  examples  in  this  dissertation. 

20  Due  to  difference  in  software,  the  GAMS  method  of  function  calls  was  used.  Therefore,  function 
calls  for  both  deterministic  and  robust  cases  are  provided  so  the  reader  can  compare  solutions. 
Computational  times  have  also  been  provided.  For  a  discussion  on  function  calls,  please  refer  to 
Appendix  B. 
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Table  3.1:  Analysis  of  function  calls  for  one  iteration 


Operation 

Number  of 

Function  Calls 

Assignments 

Objective  Function 

1 

1 

Iteration  Counter 

1 

1 

Constraints 

J+l 

N(J+1) 

( Extra  1  for  obj.  robust) 

Fixing  Uncertainty 

N 

N 

at  Lower  Level 

Slope  of  Modified 

N 

N(J+1) 

Benders  Cut 

Sample  Points  for  Nonlinear 

2K 

2KN  +  4KN-2N 

Total  Maximum  Expected 

~ 

2 +  N  +  JN+  6KN 

For  most  of  the  numerical  problems  in  this  chapter,  K  is  at  most  10.  Hence, 
the  number  of  binary  variables  introduced  to  the  fonnulation  is  NJK.  Note  that  this  is 
the  maximum  theoretically  possible  function  calls  and  actual  function  calls  (as  shown 
by  examples)  are  much  less.  For  example,  in  the  heat  exchanger  problem  (Section 
3.5.3),  N  =  8,  J  =  Y1,K=  10  and  6  iterations  were  used  to  solve  the  problem  so  the 
maximum  expected  number  of  function  calls  is  6(2  +  8  +  (17)(8)  +  6(10)(8))  =  3,756. 
According  to  the  results,  it  took  only  984  to  solve  this  particular  problem. 
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3.4.  Numerical  Results 


The  following  numerical  and  engineering  examples  serve  to  demonstrate  applicability 
of  the  algorithm,  compare  the  proposed  algorithm  to  a  previous  algorithm  from  the 
literature,  and  show  the  different  types  of  problems  that  can  be  solved.  The  first 
example  is  a  simple  quadratic  program  to  show  the  algorithm  steps  in  detail.  The  next 
three  examples  then  increase  the  complexity  (number  of  variables,  nonlinearity)  of 
this  quadratic  program  and  show  how  the  number  of  function  calls  changes.  The  next 
two  examples  are  similar  linear  programs  except  that  Example  6  has  been  shown  to 
need  a  significantly  higher  number  of  function  calls  to  solve  for  the  locally  robust 
optimal  solution  than  Example  5  (Li  et  al.,  2011).  Examples  7  and  8  are  robust 
optimization  problems  with  quasiconvex  constraints  which  the  modified  Benders 
method  is  shown  to  solve  exactly.  The  next  four  problems  are  scalable  versions  of  an 
engineering  example  with  quasiconvex  constraints.  These  examples  show  that  the 
modified  Benders  method  is  scalable  and  can  be  applied  to  large  problems  without  a 
drastic  increase  in  function  calls.  The  final  two  examples  are  from  engineering  design 
and  are  nonlinear  (non-convex)  programs.  Of  the  two  engineering  examples,  the  first 
one  (Welded  Beam  Design)  considers  objective  robustness  and  the  second  (Heat 
Exchanger  Design)  considers  feasibility  robustness.  All  optimization  problems 
correspond  to  minimizing  a  single  objective  function  with  a  set  of  constraints. 
Problems  labeled  as  “self’  have  been  designed  by  the  author  to  use  as  test  problems 
(Siddiqui  et  al.,  2011a);  detailed  fonnulations  as  well  as  further  characteristics  of  the 
solution  are  in  Appendix  A.  Solutions  were  checked  by  a  simple  uniform 
discretization  of  the  uncertainty  range  (each  point  separated  by  0.01)  to  see  if  any  of 
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the  constraints  were  violated  under  uncertainty.  Tolerance  ( tol )  was  set  to  0.00001 
for  all  examples. 

3.4.1.  Numerical  Example  (Example  1)  to  Show  Methodology  Step-by- 
Step 

A  simple  numerical  example  is  presented  to  show  how  Benders  algorithm  is  modified 
to  obtain  robust  solutions  using  heuristic  Algorithm  3.2.  In  the  following  robust 
problem,  uncertainty  is  only  in  the  constraint  (without  loss  of  generality). 

min  ((x,  -  0.6)2  +  (x2  -  0.6)2 ) 

S.t. 

(-l  +  x)  +  Xj  +  x2  <  0 

-x,  <0  (3.21) 

~x2  <0 

where 

Vx  e  [-  Ax,  Ax] 

The  robust  solution  to  this  problem  (verified  as  unique  algebraically)  is 
Xj  =0.45, x2  =0.45  with  the  globally  interval-optimal  value  x  =  0.1.  Since  the 
constraint  functions  are  linear,  hence  quasiconvex  within  the  uncertainty  interval, 
refonnulate  as  in  (3.11)  to  have  uncertainty  variables  only  have  endpoint  values. 

minffxj  -  0.6)2  +  (x2  -  0.6)2 ) 

X  ' 

S.t. 

(— 1  +  x)  +  Xj  +x2  <  0 

-x,  <0  (3.22) 

-x2  <0 

where 

Vx  e  {-  Ax,  Ax} 

Proceed  according  to  the  modified  Benders  method  described  in  Section  3.3.2. 
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Step  1:  The  master  problem  is  the  following: 


max  a 

a,x 


0  <  x  <  1 


a  <  1000 


(3.23) 


The  upper  bound  on  a  is  chosen  to  be  large  enough  to  not  interfere  with  given  the 
form  of  (3.22).  Solving  the  above  problem  gives  a  =  1000  and  x  =  -0.1.  Here 

a  =  min^Xj  -  0.6)2  +  (x2  -  0.6)2 ) 

S.t. 

(—1  +  x)  +  Xj  +  x2  <  0  ^  24) 

-Xj  <  0 

-x2  <  0 

x  e  {-0.  l,0.l} 


Step  2:  Fix  x  and  then  solve  the  following  subproblem: 

w  =  min  ((x,  -  0.6)2  +  (x2  -  0.6)" ) 

X  ' 

S.t. 

(— 1  +  x)  +  Xj  +X2  <  0 
“Xj  <  0 
-X2  <  0 
x  =  -0.1 

This  gives:  x:  =0.55,x2  =0.55. 


(3.25) 


Step  3 :  Check  for  convergence  with  zup  -zlow  =  1000-0.005  =  999.995  where 
z/OK,  =  ((0.55 -0.6)2  + (0.55 -0.6)2)  and  zup  =  a .  Since  this  is  not  good  enough  for 
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convergence  when  compared  with  the  preselected  tolerance,  a  modified  (robust) 
Benders  cut  is  added. 


Step  4:  Add  the  following  robust  Benders  cut. 


0.005  + 


1000-0.005 
-0.1 -(0.1) 


(x-(-0.1))  >  a 


(3.26) 


Step  1  (returned):  Solve  the  following  master  problem  after  adding  the  robust  Benders 
cut: 


min  a 

X 


S.t. 

-0.1  <x<0.1 
a  <  1000 


0.005  + 


1000-0.005 

—  0.1  — (0.1) 


(x  -  (-0. 1))  >  a 


(3.27) 


Solving  the  above  problem  gives  a  =  1000  and  x  =0.1.  Then  go  back  to  Step  2  and 
solve  the  subproblem  with  x  =  0.1  fixed.  A  new  modified  Benders  cut  will  now  be 
added. 


The  following  graph  (Figure  3.4)  shows  what  happens  when  this  cut  is  added. 
The  standard  Benders  decomposition  method  would  have  taken  a  cut  that  would  have 
forced  x  =  -0.1  and  that  would  have  given  the  constraint  with  the  dashed  line. 
However,  the  robust  Benders  cut  generates  the  cut  signified  by  the  dotted  line,  which 
is  in  fact  the  constraint  that  forms  the  border  of  the  robust  feasible  region. 
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Figure  3.4:  Adding  a  modified  (Robust)  Benders  Cut 


The  algorithm  proceeds  in  this  manner  until  convergence.  Table  3.2  summarizes  these 
results. 


Table  3.2:  Solution  Steps  for  Modified  Benders  Approach 


Iteration 

X 

Xl 

X2 

Zlow 

Zup 

1 

-0.1 

0.55 

0.55 

0.005 

1000 

2 

0.1 

0.45 

0.45 

0.045 

1000 

3 

0.1 

0.45 

0.45 

0.045 

0.045 
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This  simple  problem  was  solved  in  three  iterations.  The  final  row  corresponds 
to  the  globally-optimal  robust  solution,  which  can  be  verified  algebraically  to  be 


globally  robust  optimal.  The  details  are  shown  in  the  following  table  (Table  3.3). 


Table  3.3:  Detailed  Solution  for  Simple  Problem 


Information 

Nominal  Solution 

Robust  Solution 

Xl 

0.5 

0.45 

X2 

0.5 

0.45 

fix) 

0.02 

0.045 

Function  Calls 

5 

11 

3.4.2.  Numerical  Results 

Table  3.4  describes  the  results  obtained  from  the  numerical  test  problems.  The  first 
two  examples  have  only  uncertainty  in  the  parameters  while  the  rest  have  uncertainty 
in  the  parameters  and  the  decision  variables. 

The  same  test  problems  were  solved  using  Li  et  al.”s  (Li  et  ah,  2006)  method 
for  robust  optimization  and  the  results  are  displayed  in  Table  3.4.  Not  only  does  Li  et 
al."s  (Li  et  ah,  2006)  method  use  a  lot  of  function  calls,  but  often  the  robust  function 
value  is  higher  than  the  modified  Benders  value.  Once  the  number  of  function  calls 
exceeded  109,  the  run  was  stopped.  These  solutions  have  not  been  reported  in  Table 
3.4  as  well  as  in  the  rest  of  the  chapter  as  they  were  infeasible  in  the  solver  (which 
can  imply  that  the  problem  was  too  computationally  intensive  for  the  solver). 
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3.5.  Engineering  Design  and  Other  Applications 


3.5.1.  Fleury’s  Weight  Minimization 

This  is  a  modified  example  from  the  literature  (Groenwold  &  Etman,  2010)  so  that 
interval  uncertainty  is  present  in  all  the  decision  variables.  This  example  supports  the 
approach  for  feasibility  robustness  as  well  as  corroborates  the  fact  that  the  modified 
Benders  method  is  able  to  tackle  problems  with  large  number  of  variables  and 
constraints  without  being  computationally  expensive.  For  N  variables,  the  problem  is 
as  follows: 


TV 

min  /{x)  =  Yjxi 


i= 1 


S.t. 


0.95  N 

z 


l  l 


N  1 

z  — 


M  Xt  +  Xt  N  I=0.95JV+1  xi  +  X, 


-  N  <0 


0.957V  i  i  TV  1 

y - —  y - o.9n  <  o 

7=1  Xf  +  X .  N  7=0.957V+1  Xt  +  xt 

— ^  <  x,.  +  xt  <  N2  i  =  l,2,...,N 
N 


X e 


[-0.1,+Q.l] 


(3.28) 


The  modified  Benders  method  solved  this  problem  with  the  results  shown  in 
Table  3.5  for  N  =  1 02,  103,  1 04,  and  105.  Note  that  the  number  of  function  calls 
increases  linearly  with  the  complexity  of  the  problem.  N  represents  the  number  of 
variables  in  the  problem  and  all  of  them  have  uncertainty.  Again,  this  example  was 
compared  to  (Li  et  ah,  2006)  as  shown  in  Table  3.5.  However,  the  results  for  (Li  et 
al.,  2006)  are  not  reported  as  the  problem  was  stopped  after  a  certain  number  of 
function  calls  given  in  Table  3.5.  Here,  Li  et  al."s  (Li  et  al.,  2006)  method  could, 
conceptually,  solve  this  problem  but  would  have  taken  a  lot  of  computation  time. 
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However,  the  modified  Benders  method  solved  all  cases  and  produced  only  a  linear 
increase  in  computational  effort. 


Table  3.5:  Results  for  Fleury’s  Weight  Minimization  Like  Problem 


Number  of  Variables 

N  =  10" 

N  =  10" 

N  =  104 

N  =  10" 

Tolerance 

1CT5 

Ter8 

To7™ 

To70 

x\  to  a'o.95,y  (Determ.) 

1 

l 

1 

l 

X|  to  xo.95;y  (Robust) 

1.1556 

1.1556 

1.1556 

1.1556 

a'oy)9 y'+  i  to  xN  (Detenn.) 

W1 

ftr5 

10’4 

To75 

a'o.95 y'+  i  to  xN  (Robust) 

0.1+10’" 

0.1+10’" 

0.1+10’4 

0.1+10° 

Function  Value  (Determ.) 

95.00005 

950.0005 

9500.005 

95000.05 

Function  Value  (Robust) 

110.2820 

1102.820 

11028.20 

110282.0 

Function  Calls  (Detenn.) 

506 

2.0  x  10" 

2.0  x  104 

2.0  x  10" 

Function  Calls  (Robust) 

744 

2.3  x  104 

1.9  x  10" 

1.9  x  10b 

Fn.  Calls  (Li  et  ah,  2006) 

>RP 

>KP 

>1012 

>1012 

3.5.2.  Design  of  a  Welded  Beam 

This  example  is  a  well-known  welded  beam  problem  from  (Ragsdell  &  Phillips, 
1976).  In  this  problem,  a  beam  A  is  to  be  welded  to  a  rigid  support  member  B.  The 
beam  has  a  rectangular  cross-section  and  is  to  be  made  out  of  steel.  The  beam  is 
designed  to  support  a  force  F  =  6000  LBF  acting  at  the  tip  of  the  beam,  and  there  are 
constraints  on  the  shear  stress,  nonnal  stress,  deflection,  and  buckling  load  on  the 
beam.  The  problem  has  four  continuous  design  variables,  and  they  are:  thickness  of 
the  weld  (/?),  length  of  the  weld  (/),  thickness  of  the  beam  (/),  and  width  of  the  beam 


77 


(, b ).  All  variables  are  in  inches.  The  objective  of  the  problem  is  to  minimize  the  total 
cost  f[x)  of  making  such  an  assembly.  For  complete  fonnulation  of  the  robust 
optimization  problem  including  specific  values  of  the  parameters,  please  refer  to 
(Gunawan  &  Azarm,  2004).  Figure  3.5  shows  the  structure  of  the  beam. 


Figure  3.5:  Design  of  a  Welded  Beam  (Gunawan  &  Azarm,  2004) 

The  following  is  the  fonnulation  for  the  welded  beam  as  outlined  in  (Gunawan  & 
Azarm,  2004).  The  objective  function  is  given  by 

min  /cost  =0+G  )^1  +  CML  +  0  (3  -29) 

The  constraints  and  other  equations  are  described  below. 
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((1  +  c3)h2I  +  c4tb(L  +  /)  +  0.2 c5l)~  ((1  +  c3)/z2/  +  c4tb(L  +  /))<  0.3 

Si  =~  1-0 

—  -1<0 
°d 

g>_0.25  1S° 

S4=f-1^0 

rc 

g,  =  -*-iso 
b 

g^°;25-.,o 

h 

0.1  <  A  <  2.0 

0.1  </  <  2.0 

0.1  <  t  <  2.0 

0.1<h<2.0 

c3e  [0.0547,0.1547] 
c5  e  [0,0.25] 

where 

C3  =  cost  of  weld  material  ($0. 1047/inch3) 
c\  =  cost  of  weld  material  ($0.048 1/inch3) 

77=  maximum  shear  stress  in  weld  (psi) 

}jd  =  allowable  shear  stress  in  weld  (13,600  psi) 

C  =  maximum  normal  stress  in  beam  (psi) 

Cd  =  allowable  nonnal  stress  in  beam  (30,000  psi) 

S  =  deflection  at  beam  end  (inch) 

Pc  =  allowable  buckling  load  (LBF) 

L  =  Length  of  unwelded  beam  (14  inch) 

G  =  12  x  106  psi 
£  =  30  x  1 06  psi 

The  following  equations  are  used  to  calculate  the  above  variables. 
t  =  +2t't''cosO  +  (t")2 


(3.30) 
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F 


T  = 


4ihV 


T  = 


MR 
J  ' 


cos  6  =  — ; 
2  R 


(T  n 

1  „  l2  | 

fh  +  t) 

M  =  F 

j  R  —  ,  — H 

l  2  J 

1’  V4 

v  2  j 

(3.31) 


(3.32) 


J  =  2\  0.707/?/ 


ri2  Ah  +  t^ 
12  v  2  j 


6FL  _  AFL3 

(j  —  — r-;  o — — r — 

bt2  Et’b 


P  = 


r4M3y[ElaY{__^_  |eP 

2  L\  a  j 


L~ 


I  =  —tb2;  a  =  -Gtb 3 
12  3 


(3.33) 

(3.34) 

(3.35) 

(3.36) 


The  solution  from  the  modified  Benders  method  is  different  from  (Gunawan 
&  Azarm,  2004).  First,  the  modified  Benders  method's  nominal  solution  is  closer  to 
an  actual  solution  from  an  earlier  paper  by  (Ragsdell  &  Phillips,  1976)  who  provided 
an  optimal  objective  function  value  of  /=  2.38  while  (Gunawan  &  Azarm,  2004) 
provided  /=  2.39.  Second,  the  robust  solution  is  also  lower  in  function  value  but  still 
feasible.  The  robust  solution  is  also  feasible  for  all  realizations  of  uncertainty,  hence 
it  is  better  than  (Gunawan  &  Azarm,  2004)  reported  solution. 

This  example  highlights  the  strength  of  the  modified  Benders  method  over 
previous  methods.  Gunawan  and  Azarin's  (Gunawan  &  Azarm,  2004)  method 
involves  a  backward  mapping  approach,  which  is  known  to  omit  solutions.  The 
modified  Benders  method,  while  giving  a  better  solution  (lower  in  function  value),  is 
also  computationally  less  expensive.  The  solution  from  the  modified  Benders  method 
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was  checked  for  robustness  using  a  genetic  algorithm.  The  solution  is  displayed  in 
Table  3.6. 


Table  3.6:  Results  of  Welded  Beam  Example 


Info 

Gunawan  and 

Modified 

Gunawan  and 

Modified 

Azarm  (2004) 

Benders 

Azann  (2004) 

Benders  Robust 

Nominal  Sol. 

Nominal  Sol. 

Robust  Sol. 

Sol. 

h 

0.241 

0.2444 

0.246 

0.2392 

I 

6.158 

6.2186 

5.461 

5.6753 

t 

8.5 

8.2915 

9.138 

9.1225 

b 

0.243 

0.2444 

0.248 

0.2392 

Ax) 

2.39 

2.3807 

2.48 

2.4236 

Function 

N/A 

8 

250 

38 

Calls 

3.5.3.  Heat  Exchanger  Design 

The  energy  balance  on  a  heat  exchanger  can  be  written  as 

Q  =  UAFATm=(mcp)h(Thl-T,J  =  (mcp)c(Tc2-Tcl)  (3.37) 

Several  equations  govern  the  above  heat  transfer.  The  above  equation  (3.37)  will  be 
used  as  an  objective  function  that  is  to  be  maximized,  as  well  as  constraints  that 
restrict  the  structure,  in  particular  constraints  on  the  pressure  drop  on  the  tube  side 
( A pt )  and  shell  side  ( A ps ).  Subscript  1  denotes  the  fluid  entering  while  subscript  2 
denotes  it  leaving;  c  denotes  the  cold  fluid  and  h  the  hot  fluid.  In  this  example,  cold 
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water  is  in  the  tubes  and  hot  water  is  on  the  shell  side  and  the  problem  has  been  set  up 
in  a  counterflow  arrangement  for  a  124  tubes  and  two-pass  heat  exchanger.  The 
following  lists  the  important  variables  and  parameters  considered  in  this  design. 


Symbol 

Units 

Description 

Value 

Cp 

J/(kgK) 

Specific  heat  at  constant  pressure 

Variable 

do 

m 

Tube  outside  diameter 

Variable 

f 

Tube  flow  friction  factor 

Variable 

fs 

Friction  factor  shell  side 

Variable 

ho 

W/(m2  K) 

Heat-transfer  coefficient  outside  tube 

Variable 

h 

W/(m2  K) 

Heat-transfer  coefficient  inside  tube 

Variable 

k 

W/(m  K) 

Thermal  conductivity  of  fluids 

Variable 

dps 

Pa 

Shell-side  pressure  drop 

Variable 

Apt 

Pa 

Tube-side  pressure  drop 

Variable 

u, 

M/s 

Mean  axial  velocity  of  fluid  in  tube 

Variable 

Ao 

m2 

Tube  outside  surface  area  per  pass 

Variable 

Ai 

m2 

Tube  inside  surface 

Variable 

As 

m2 

Cross-flow  area  at  or  near  shell  centerline 

Variable 

A, 

m2 

Total  cross-sectional  area  of  tubes  per  pass 

Variable 

B 

m 

Baffle  spacing 

0.5 

C 

m 

Clearance  between  adjacent  tubes 

Variable 

CL 

Tube  layout  constant 

1  (for  90°) 

Ctp 

Tube  count  calculation  constant 

0.90 

De 

m 

Equivalent  diameter  of  shell 

Variable 

F 

LMTD  correction  factor 

Variable 

L 

m 

Tube  length 

Variable 

Nb 

Number  of  Baffles  (Integer  =  B/L) 

4 

Nt 

Number  of  Tubes 

124 

NP 

Number  of  Tube  passes 

2 

Pp 

W 

Pumping  power  of  fluid  in  tubes 

Variable 

Pr 

Prandtl  Number 

Variable 

Q 

W 

Heat-transfer  rate 

Variable 

Rfo 

(m2  K)/W 

Fouling  resistance  on  outside  of  tube 

0.00015 

Rfi 

(m2  K)/W 

Fouling  resistance  on  inside  of  tube 

0.00015 

Reh 

Reynolds  number  at  Tb 

Variable 

Res 

Shell-side  Reynolds  number  at  Tb 

Variable 

ATm 

K 

LMTD 

Variable 

Th2 

K 

Outlet  temperature  of  hot  fluid 

Variable 

Tc2 

K 

Outlet  temperature  of  cold  fluid 

315 

Tb 

K 

Bulk  temperature 

Variable 

Tw 

K 

Wall  temperature 

Variable 

U 

W/(m2  K) 

Average  overall  heat  transfer  coefficient  based  on  A 

Variable 

Os 

Viscosity  correction  factor 

Variable 

p 

kg/(s  m) 

Dynamic  Viscosity 

Variable 

fib 

kg/(s  m) 

Dynamic  Viscosity  at  Tb 

Variable 

Mw 

kg/(s  m) 

Dynamic  Viscosity  at  Tw 

Variable 

P 

kg/m3 

Density 

Variable 
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The  following  equations  are  the  ones  coded  into  MATLAB  and  are  selected  from  the 
whole  formulation  to  provide  further  insight.  For  the  complete  formulation  and  all 
equations  used,  please  refer  to  (Magrab  et  ah,  2004). 


Tt,=T„-(^^-(Tci-Ta) 

(mcp)s 

(3.38) 

A p,=C„ 

UFAT 

m 

(3.39) 

.  fsms\Nb+\)Ds 

2  As2pDec/>s 

(3.40) 

L_(mcp)t(Tc2~Tcl) 

(3.41) 

nd{)N  T  FAT m 

c  _Np(cp\(Tc2-Tcl) 

Ixd^oNjP'A,2 

(3.42) 

.  7vd~NT 

A,  = — — - 
'  4  NP 

(3.43) 

Table  3.7  lists  the  design  variables  and  parameters  with  uncertainty. 
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Table  3.7:  Design  Variables  and  Parameters  with  Uncertainty 


Symbol 

Unit 

Description 

Uncertainty 

di 

m 

Tube  inside  diameter  (Variable) 

dj  ±  0.001 

m, 

kg/s 

Tube-side  mass  flow  rate  (Variable) 

mt  ±  1 

ms 

kg/s 

Shell-side  mass  flow  rate  (Variable) 

ms  ±  1 

A 

m 

Shell  inside  diameter  (Variable) 

Ds  ±  0.01 

Pt 

m 

Pitch  size  (Variable) 

PT  ±  0.01 

Phi 

K 

Inlet  temperature  of  hot  fluid 

65  ±  1 

Pci 

K 

Inlet  temperature  of  cold  fluid 

18  ±  1 

ktube 

W/(m  K) 

Thermal  conductivity  of  tubes 

60  ±  1 

Figure  3.6:  Heat  Exchanger  Schematic  (Magrab  et  al.,  2004) 


The  optimization  problem  is  the  following: 
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max  Q 
s.t. 

A ps  <  35000 
A pt  <  50000 
Th2  <  60 

NpAt  <  As 

-  C  <  0  (3.44) 

Z  <  40 

7  <mt  +  mt  <  10 
13  <  ms  +  ms  <  16 
0.001  <  Ds  +  D  <  2 
0.002  <  PT  +PT  <0.1 
0.0001  <  di+di  <0.1 

The  following  table  shows  the  results. 


Table  3.8:  Results  for  Heat  Exchanger  Design 


Variables 

Nominal  Solution 

Robust  Solution 

1006.77 

906.09 

di 

0.0160 

0.0149 

m, 

10 

9 

ms 

14 

14 

Ds 

0.3900 

0.3900 

Pt 

0.0240 

0.0311 

Function  Calls 

49 

984 

The  thing  to  note  about  this  example  is  that  with  less  than  an  average  of  1% 
uncertainty,  the  objective  function  value  decreases  by  almost  10%.  Hence,  in  the 
design  of  any  model,  it  is  important  to  consider  the  uncertainty  in  the  problem,  which 
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can  lead  to  different  designs  as  well.  This  problem  was  also  tried  with  Li  et  al."s 
(2006)  method  however  after  109  function  calls  without  convergence,  the  approach 
was  stopped. 

For  completeness,  Table  3.9  displays  actual  computational  time  for  each  test 
problem  as  well. 

Table  3.9:  Number  of  Iterations  and  CPU  Time  to  Solve  Problems 


Test 

Number  of 

CPU  (2.0  GHz,  4GB  RAM) 

Problem 

Iterations 

Time  (s) 

Example  1 

3 

0.560 

Example  2 

3 

0.787 

Example  3 

3 

1.654 

Example  4 

3 

1.435 

Example  5 

3 

3.821 

Example  6 

3 

3.494 

Hock  100 

4 

30.552 

Hock  106 

4 

25.645 

Fleury  (N=10“) 

6 

9.328 

Fleury  (N=10J) 

9 

324.532 

Fleury  (N=104) 

12 

886.321 

Fleury  (NMO5) 

15 

2123.453 

Welded  Beam 

4 

1.606 

Heat  Exchanger 

6 

45.234 
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3.5.4.  Building  Energy  Intensive  Infrastructure 


This  example  takes  a  problem  of  a  decision  maker  to  decide  whether  to  build  energy 
intensive  infrastructure  at  intensity  H  when  there  is  uncertainty  in  future  carbon  tax 
and  retrofit  cost.  This  example  is  a  modified  version  of  the  formulation  from  the 
paper  (Strand  et  ah,  2011).  Investments  in  large,  long-lasting,  energy-intensive 
infrastructure  that  use  fossil  fuels  increase  longer-term  energy  use  and  greenhouse  gas 
emissions,  unless  the  plant  is  shut  down  early  or  undergoes  costly  retrofit  later.  These 
investments  will  depend  on  expectations  of  retrofit  costs  and  future  energy  costs, 
including  energy  cost  increases  from  tighter  controls  on  carbon  emissions. 

Consider  a  decision  maker  in  a  world  with  two  periods.  Infrastructure 
investment  is  made  at  the  start  of  period  1,  and  can  be  “retrofitted”  at  the  start  of 
period  2.  As  long  as  it  is  operated  and  not  retrofitted,  a  given  infrastructure  gives  rise 
to  a  given  energy  consumption  per  unit  of  time,  detennined  at  the  time  of  initial 
investment.  Energy  supply  costs  and  environmental/climate -related  costs  are 
uncertain  at  the  time  of  establishment  in  period  1,  but  are  revealed  at  the  start  of 
period  2.  Assume  both  periods  have  the  same  length,  and  there  is  no  discounting 
within  the  periods.  The  problem  of  a  decision  maker  is  given  by 

max  U(H )  -  txH  +  (U(H)  -t2( l-  a)H  -  raH ) 

H  ,a 

S.t. 

0  <  a  <  1  (3.45) 

0  <H< 

t2 (1  -  a)H  +  raH  <  U(H )  -  txH 

Here,  U(H)  is  the  utility  of  the  decision  maker  when  selecting  an  energy 
investment  intensity  H.  The  costs  (carbon  tax,  for  example)  in  the  first  period  for  this 
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energy  are  t\  and  in  the  second  period  are  t2.  The  ratio  a  is  the  amount  of  energy 
investment  that  is  retrofitted,  and  r  is  the  cost  of  that  retrofit.  In  (3.45),  the  decision 
maker  aims  to  maximize  utility  in  the  two  periods  (without  any  discounting). 

As  in  (Strand  et  ah,  2011),  the  uncertainty  is  present  in  the  values  of  and  r. 
The  last  constraint  in  the  above  fonnulation  makes  sure  that  the  cost  for  retrofitting 
and  paying  carbon  tax  in  the  second  period  is  below  the  excess  utility  achieved  in  the 
first  period.  For  the  numerical  study,  the  following  parameters  were  chosen. 

•  U(H)  =  8H-H 2 

.  =  4 

•  h  =  1 

•  t2  e  [4  —  At2,4  +  At,] 

•  re  [5. 5, 6.5] 

Note  that  in  this  first  case,  we  have  assumed  that  ti  has  uncertainty  of 
magnitude  A  tj  while  uncertainty  in  retrofit  cost  r  is  given  as  above.  The  goal  is  to  see 
what  happens  as  this  uncertainty  range  is  increased.  The  robust  optimization  problem 
to  solve  is 

max  U(H)  -  txH  +  ( U(H )  -t2(  1  -  a)H  -  rccH) 

H,a 

S.t. 

0  <  a  <  1 

0  <H<H™X  (3.46) 

t2  (1  -  a)H  +  raH  <  U(H)  -  txH 

V 12  e  [4  -  A t2 ,4  +  At 2  ] 

Vr  e  [5. 5, 6. 5] 
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The  modified  Benders  method  is  applied  to  solve  this  problem.  Table  3.10  shows 
what  happens  to  energy  intensity  H  and  selection  of  retrofit  vs.  not  retrofit  as  At? 
increases. 

Table  3.10:  Results  for  Increasing  Uncertainty  in  t2 


Value  of  At2 

Energy  Intensity  (H) 

Retrofit?  (Value  of  a) 

0 

3 

No  (a  =  0) 

1 

2 

No  (a  =  0) 

2 

1 

No  (a  =  0) 

3 

0.5 

Yes  (a  =  1) 

4 

0.5 

Yes  (a  =  1) 

Note  that  with  increasing  uncertainty,  the  decision  maker  chooses  less  energy 
intensive  infrastructure.  This  is  at  odds  with  the  probability-based  analysis  done  in 
(Strand  et  al.,  2011).  Since  (Strand  et  ah,  2011)  assumed  probability  distributions  for 
?2  and  r,  an  increase  in  uncertainty  meant  a  good  chance  that  ti  would  offer  a  low  tax 
in  the  future  as  well.  Hence,  an  increase  in  uncertainty  brought  an  increase  in  energy 
intensive  investment.  One  of  the  main  reasons  this  answer  is  different  is  that  robust 
optimization  considers  a  worst-case  analysis.  Hence,  with  increasing  uncertainty,  the 
extremely  risk-averse  robust  optimizer  chooses  a  progressively  safer  option,  to  avoid 
any  chance  of  not  being  able  to  afford  retrofit  or  tax  in  the  future.  Hence,  robust 
optimization  gives  us  an  alternative  way  to  analyze  this  problem. 
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3. 6.  Summary 


This  chapter  presents  an  efficient  robust  optimization  approach  to  solve  problems  that 
have  parameters  and/or  decision  variables  with  interval  uncertainty.  The  proposed 
modified  Benders  method  obtains  robust  optimal  solutions  to  linear  programming, 
quadratic  programming,  convex  and  non-convex  programming  problems.  The 
approach  is  computationally  tractable  and  is  tested  with  14  numerical  and  engineering 
examples  with  the  most  general  being  nonlinear  (non-convex)  objective  function  and 
nonlinear  (non-convex)  constraint  robust  optimization  problems.  The  modified 
Benders  method  provides  an  approximate  locally  optimal  robust  solution  to  general 
nonlinear  robust  optimization  problems,  with  a  way  to  improve  this  approximation  if 
desired. 

The  test  examples  show  the  strength  of  this  method  when  compared  to  two 
previous  approaches.  Not  only  is  the  method  computationally  efficient,  but  also 
obtains  better  solutions  when  compared  to  these  previous  methods.  The  method  is 
scalable,  that  is,  number  of  function  calls  increases  at  most  linearly  with  an  increase 
in  number  of  variables  for  the  problems  tested. 
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Chapter  4:  Solving  Mathematical  Programs  and 


Equilibrium  Programs  with  Equilibrium  Constraints 

4.1.  Introduction 

This  chapter  describes  a  new  algorithm  to  solve  mathematical  programs  and 
equilibrium  programs  with  equilibrium  constraints.  Numerical  examples  are  provided 
in  each  case,  along  with  a  test  of  computational  time  with  disjunctive  constraints.  An 
application  of  the  method  to  a  large-scale  North  American  gas  market  model  is  also 
provided. 

The  motivation  behind  developing  an  algorithm  for  MPECs  was  to  find  an 
alternative  to  traditional  techniques,  in  particular  disjunctive  constraints.  This  was 
necessitated  by  the  need  to  solve  large-scale  MPECs  representing  natural  gas  markets. 
A  North  American  gas  model  was  developed  from  the  larger  World  Gas  Model 
(Gabriel  et  ah,  2011c).  Various  techniques"  were  employed  to  solve  this  North 
American  gas  model,  but  the  only  successful  one  was  the  application  of  Algorithm 
4.1  presented  later  in  this  chapter. 

Hence,  for  this  research  thrust,  the  application  drove  the  theory.  In  this 
chapter,  the  current  state  of  the  theory  is  presented.  As  these  ideas  are  still  being 


21  The  fine  tuning  of  solvers  was  needed  to  be  able  to  solve  the  North  American  gas  model.  In 
particular,  the  SBB  solver  (GAMS,  2010)  was  used  in  conjunction  with  CONOPT  (GAMS,  2010).  The 
iteration  limits  for  the  first  search  of  SBB  needed  to  be  increased.  For  particular  sections  of  the  branch 
and  bound  tree,  a  breadth-first  approach  was  employed. 
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developed,  there  is  great  room  for  further  development,  including  the  EPEC  solution 
techniques. 

The  chapter  starts  with  a  general  theory  of  the  new  MPEC  and  EPEC 
technique  followed  by  examples.  Finally,  an  application  of  the  technique  is  presented 
for  the  North  American  gas  market  model.  A  portion  of  this  chapter  has  been 
presented  in  (Siddiqui  &  Gabriel,  2011b)  and  (Gabriel  et  al.,  2011c). 

4.2.  Solving  Mathematical  Programs  with  Equilibrium  Constraints 

4.2,1.  Changing  the  Formulation  of  the  Lower-Level  Problem 

Recall  that  approximating 

yTg(x,y)  =  0  (4.1) 

was  one  of  the  hurdles  in  solving  MPECs.  If  g  is  a  linear  tenn,  approximating  the  left- 
hand  side  of  (4.1)  often  involves  specialized  techniques,  one  of  which  happens  to  be 
Schur"s  decomposition  followed  with  an  approximation  by  linear  functions  (Gabriel 
et  al.,  2006).  Moreover,  results  in  this  dissertation  corroborate  this  fact  by  using  the 
same  idea  for  a  vector-valued  linear  function  g  if  linear  constraints  are  included.  First, 
using  Schur"s  decomposition,  vectors  u  and  v  (dependent  on  x  and  y )  are  used  to 
rewrite  the  original  MPEC  (2.7)  as 
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min  f(x,y) 
s.t.(x,y)  e  Q 
y>0 

g(x,y)>  0  (4.2) 

uTu-vTv  =  0 
y+g(x,y) 

U  — 

2 

v  =  y-s(x,y) 

2 

Now,  the  optimization  problem  does  not  contain  any  bilinear  terms.  In  fact 

uTu  —  vTv  =  0  (4.3) 

can  be  readily  approximated  using  SOS  type  2  (Beale,  1975)““  variables  to  create  a 

piecewise-linear  function.  However,  realizing  that  the  complementarity  conditions 

force  y  >  0  and  g(x,y)  >  0 ,  shows  that  only  the  positive  square  root  of  u  will  give  a 

feasible  solution  to  the  problem.  Hence,  (4.2)  can  be  refonnulated  as  (4.4)  below 

min  f(x,  y) 
s.t.(x,y)  e  Q 
y>0 

g(x,y)>  0  (4.4) 

u  -  |v|  =  0 
..  _  y  +  g(x,y ) 

Li  — 

2 

v_y-g(x,y) 

2 

The  next  theorem  shows  that  the  solution  sets  to  (2.5),  (2.16),  and  (4.4)  are  the  same. 


22  Special  ordered  sets  of  type  2  (SOS  type  2)  variables  are  defined  as  a  set  of  positive  variables  of 
which  at  most  two  can  be  non-zero,  and  if  two  are  non-zero  then  they  need  to  be  next  to  each  other. 


93 


Theorem  4.1.  Let  the  solution  set  to  formulation  (2.5)  be  given  by  Si,  the  solution 
set  to  formulation  (2.16)  be  given  by  S2  and  the  solution  set  to  formulation  (4.4) 
be  given  by  S3.  Then,  given  a  large  enough  value25  of  K,  Si  =  S2  =  S3. 

Proof.  Realize  that  all  three  formulations  (2.5),  (2.16),  and  (4.4)  have  the  same 
objective  function.  Hence,  it  is  sufficient  to  show  that  all  three  formulations  have  the 
same  feasible  region.  Hence,  assume  that  Si,  S2,  S3  represent  the  feasible  regions  of 
formulation  (2.5),  (2.16),  and  (4.4),  respectively.  We  will  show  these  feasible  regions 
are  equivalent  by  showing  Sj  c=  S2  c=  S3  c=  <Sj .  The  subscript  i  will  denote  vector 
element  computation. 

Pick  a  point  (x\yl)  e  Sl.  We  want  to  show  that  there  exists  a  value  of  r  such 
that  (x1 ,  y 1 )  e  S2 .  Then,  for  all  i,  either  y}=0,  or  g.(x1,y1)  =  0  or  both.  Suppose 
yj  =  0 .  Then,  in  fonnulation  (2.16),  let  r,  =  1,  which  implies  y\  =  0  in  formulation 
(2.16)  as  well.  Ifg^x1,^1)  =  0,  then  choose  rt  =  0,  which  implies  g!(x1,y1)  =  Oin 
formulation  (2.16)  as  well.  If  both  are  zero,  choose  r,  =  1  (or  r,  =  0),  which  will  ensure 
that  yj  =0and  that  g^x1,^1)  =  Ois  within  the  feasible  region  of  (2.16).  Since  K  is 
chosen  to  be  large  enough,  these  arguments  imply  that  the  solution  set  to  (2.5)  is 
contained  in  the  solution  set  to  (2. 16),  i.e.,  S]  c^. 

Next,  pick  ( x2,y2,r 2)  e^x {0,l}"v  which  is  a  solution  to  (2.16).  Consider  any 
vector  element  i.  Suppose  that  rf  =  0.  This  implies  g,(x2,y2)  =  0,  which  implies 


23  So  that  Disjunctive  Constraints  provides  the  same  solution  set  as  (2.16). 
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2  2 
.2  _ y,  >  .2  _  y, 


uf  =  and  vf  =  .  Hence,  this  implies  u\ 2  -vf  =  0  ,  and  in  particular  wf  -  vf  =  0 


On  the  other  hand,  r(2  =  1  implies  y 2  =  0 ,  uf  =  ^ X  —  - ,  and  vf  =  -  ’"V  - 


Hence,  this  case  also  implies  that  uf  -  vf  =  0  .  Therefore,  (x  ,y  )  e  S3  and  S2  c:S3. 


Now  pick  any  solution  (x3,y3)  e  S3.  For  this  solution  uf  -  vf  =  Ofor  each  i. 


Hence,  this  implies  (uf )  - (|vj|)  =0  and,  in  particular  (u^J  -(v,.3)2  =0.  Then,  the 


following  argument  shows  that  S3  ci  5j . 


k7-(v/3)2 =0 


<=> 


(y-)  +  2-(T,.3)-g,.(x3,/)  +  (g,.(x3,/))2  (y?)  -2  \y: ')•  g,(x3,y3)  +  (g.(x3,y3))2 


=  0 


<=> 

v/-g,(x3,y3)  =  0. 


Hence,  Si,  S2,  and  S3  are  subsets  of  each  other  so  they  are  equivalent.! 


4.2.2.  Approximating  The  Absolute  Value  Function  Using  Special 
Ordered  Sets  of  Type  1  Variables 

The  previous  proof  shows  that  using  the  absolute  value  function  can  be  a  substitute 
for  using  disjunctive  constraints.  However,  the  absolute  value  function  is  also  a 
nonlinear  function  which  can  provide  computational  difficulty  to  optimization  solvers 
(Steffensen  &  Ulbrich,  2010).  Hence,  a  reformulation  is  required. 
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The  absolute  value  can  be  reformulated  as  in  (4.5)  using  Special  ordered  sets 
of  type  1  (SOS1)  variables  (Beale  &  Tomlin,  1970).  SOS1  variables  are  defined  as 


sets  of  non-negative  variables  of  which  at  most  one  can  be  non-zero. 


min  f(x,y) 
s.t.(x,y)  e  Q 

T>0 

g(x,y)>0 


u  —  (v+  +  v  )  =  0 

y  +  g(x,y) 


u  = 


(v+-v-)  = 


y-g(x,y ) 


where  v+ ,  v  are  SO  S 1  variables 


(4.5) 


Lemma  4.1.  Let  5*4  be  the  solution  set  to  (4.5).  The  solution  sets  to  formulation 
(4.5)  and  formulation  (4.4),  £4  and  £3  respectively,  are  equivalent.  That  is,  £4  = 

£3. 

Proof.  Again,  since  the  objective  functions  for  both  formulations  are  the  same,  it  is 
sufficient  to  show  that  both  fonnulations  have  the  same  feasible  region.  Hence, 
assume  that  £4  and  £3  represent  the  feasible  regions  of  formulation  (4.5)  and  (4.4), 

respectively.  Set  v+-v~=v.  Then,  for  all  i,  either  (v+),  =0,  or  (v")(.  =0  or  both 
because  (v+)(.  ,(v-),.  is  a  set  of  SOS1  variables  where  at  most  one  can  be  nonzero.  This 
implies  that  v+  +  =  |v|  (componentwise  absolute  value).  Hence,  we  can  substitute  v 

in  for  v+  —v  in  fonnulation  (9)  and  |v|  for  v 1  +  v  in  formulation  (4.5)  to  get 
formulation  (4.4).  The  substitution  the  other  way  works  as  well,  hence  £4  =  £3.  ■ 
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4.2.3.  Approximating  Absolute  Value  Function  Using  a  Penalty  Method 


The  SOS1  approach  at  times  can  numerically  fail  for  more  complex  problems,  as 
SOS1  variables  also  require  binary  variables  to  be  formulated  within  the  solver 
(GAMS,  2010).  For  example,  the  North  American  Gas  model  could  not  be  solved 
using  the  SOS  1  formulation  and  instead  required  a  better  starting  point  as  described 
in  Section  4.4.  The  North  American  Gas  model  was  eventually  solved  using 
Algorithm  4.1  described  in  Section  4.2.4.  Several  other  alternatives  were  explored  to 
approximate  this  absolute  value  function.  In  particular,  Steffensen  and  Ulbrich 
(Steffensen  &  Ulbrich,  2010)  provide  a  smooth  function  approximation  to  the 
absolute  value  function.  However,  their  methodology  did  not  work  when  applied  to 
the  example  (U.S.  version  of  the  World  Gas  Model  (Gabriel  et  ah,  2011c))  in  this 
chapter.  An  alternative  way  to  approximate  the  absolute  value  function  is  the  penalty 
method  (Bazaraa  et  ah,  1993),  which  works  well  for  finding  solutions  to  MPEGs. 


ny 

min  f{x,  y)  +  £  L  (v,+  +  v;7) 

i= 1 

s.t.(xj)efi 

T>0 

g(x,y)>  0 


u  -  (v+  +  v  )  =  0 
y  +  g(x,y) 


u  = 


(v+-v-)  = 


y-g(x,y ) 


where  v+ ,  v  are  non  -  negative  variables 


(4.6) 


Theorem  4.2.  Assume  that  the  Karush-Kuhn-Tucker  conditions  are  both 
necessary  and  sufficient  for  the  optimization  problem  (4.6).  If  formulation  (4.5) 


97 


has  a  solution,  then  for  any  Li  >  0  and  for  each  i,  at  most  one  of  (v+)/  and  (v  )/  is 
nonzero  in  formulation  (4.6). 

Proof.  We  will  show  this  by  contradiction.  Suppose  that  there  exists  a  Z,  >  0  such  that 
a  solution  to  (4.6)  gives  an  index  i  where  both  W  >  0  and  (v  >  0  .  Let  the 
following  be  the  slightly  altered  form  of  (4.6)  considered  where  the  Lagrange 
multipliers  are  included  in  parentheses  and  {{x,  y)  s.t.  C{x,  y)  <  0}  defines  the  set  of 
constraints  that  define  Q. 


n 

min  f(x,  y)  +  X  A  (f+  +  v," ) 
1=1 


C(x,y)<0 

(A) 

-y<0 

(A) 

-g(x,y)<0 

(A) 

(v*+v-)--V  +  S(l-’')=0 

2 

(A) 

2 

(A) 

where  v+,v  are  non  -  negat  ive  variables 


Then,  taking  the  first-order  Karush-Kuhn-Tucker  conditions  (Bazaraa  et  ah, 
1993),  respectively,  for  (v+)(  and  (v  ),  gives  L.+(X4\+(X5\=  0  and 

L.  +  (A4  ) .  -  (As  );  =  0  since  both  (v+)(.  >0  and  (v  )(  >  0 .  Together  these  two 
conditions  imply  (A5).  =  0  and  Z,  =  -(X4  )j .  This  implies  that  the  Lagrangian  (A)  can 
equivalently  be  expressed  as 
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K  =  f(x,y)+  £i,(v/  +v,')-(l4),(v;  +v,-)  +  rt)r  CUy)  +  (^f(-y) 

j=hj*‘ 


+  U,)T(-g(x,y))  +  (A 4)t  (v++v~)- 


v  +  g(x,^) 

2  y 


+  (A5)r  (v+-v-)- 


y-g(*,y) 


\ 

±±y± 

2  J 


Realizing  that  X4  now  appears  in  two  terms,  we  can  factor  this  out  and  realize 
that  the  following  optimization  problem  will  give  the  same  solution  as  fonnulation 
(4.7)  above. 


ny 

min  f(x,y)  +  £  Lf  (v/  +  v  ~  ) 

7=1  ,j*i 

C(x,y)  <  0 

(A) 

-y<0 

U2) 

~g(x,y)<  0 

w 

(v  *+v  -)-^"+Sj"(X,4)=0 

\yj*i  ^  y j±i  2 

(K 

yi+gi(.*,y)  _0 

2 

M, 

(v*_v-)_r-«(*,r)=0 

2 

Us) 

where  v+ ,  v  are  non  -  negative  variables 


But  since  L,  >  0,  this  implies  (A4 ).  <  0  and  since  the  above  fonnulation  satisfies 

24 

necessary  and  sufficient'  conditions  for  the  Karush-Kuhn-Tucker  conditions, 


formulation  (4.8)  indicates  that 


y,-+gi(x,y) 

2 


=  0  .  Since  both  y  and  g  are  constrained 


24  Necessary  conditions  are  needed  to  go  from  formulation  (4.7)  to  the  Karush-Kuhn-Tucker  conditions 
and  sufficient  conditions  to  go  from  Karush-Kuhn-Tucker  conditions  of  (4.7)  to  optimization 
formulation  (4.8).  Also,  it  can  be  argued  that  the  constraint  associated  with  {A4)j  need  not  be  an 

equality  constraint.  Hence,  we  include  the  fact  that  (A4  ).  <  0  to  ensure  that  we  get  equality  for  the 
associated  constraint. 
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to  be  nonzero,  this  implies  that  (v+);.  =  (v  =0  for  the  index  i  in  (4.7).  This  is  a 

contradiction.  Hence,  for  all  Z,  >  0,  the  fonnulation  (4.6)  gives  a  solution  where  for 
each  i,  at  most  one  of  (v+),  and  (v');-  is  nonzero.* 

From  this  point  on,  L  =  max{Z;}  will  be  the  constant  for  each  variation  of 
(4.6).  The  value  of  the  constant  Z  should  be  chosen  to  be  small  enough  so  it  does  not 
interfere  with  the  solution.  It  is  not  known  at  this  time  if  there  always  exists  a  value  of 
Z  for  which  an  exact  solution  is  achieved  but  numerical  results  suggest  there  are 
multiple  values  of  Z  for  which  a  solution  to  the  MPEC  can  be  obtained.  By  Theorem 
4.2  for  all  positive  Z,  (4.6)  provides  a  solution  that  is  always  a  feasible  solution  to 
(4.5)  but  not  necessarily  optimal  for  large  values  of  Z.  Therefore,  Z  can  be  chosen  to 
be  machine  epsilon  .  Numerical  results  validate  that  as  Z  approaches  zero,  the 
optimal  objective  function  value  of  (4.6)  approaches  the  optimal  objective  function 
value  of  (2.5).  At  times,  solvers  will  fail  to  solve  MPECs  by  finding  an  infeasible 
solution  where  there  exists  an  i,  for  which  both  of  (v+),  and  ( v  )l  are  nonzero.  An 
alternative  to  this  is  provided  by  the  following  (heuristic)  Algorithm  1 . 

4.2.4.  Algorithm  4.1  to  Solve  Mathematical  Programs  with  Equilibrium 

Constraints 

Step  0:  Pick  a  tolerance  t. 


25  Machine  epsilon  is  defined  as  the  smallest  positive  number  specific  to  the  computer,  in  this  case  10 

17 
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Step  1:  Solve  the  problem  using  the  penalty  method  fonnulation  (4.6)  with  L 


=t. 


Step  2:  Check  for  any  pairs  of  variables  v+  and  v"  that  are  both  non-zero.  If 
yes,  go  to  Step  3.  If  not,  skip  to  Step  6. 

Step  3:  Refonnulate  those  particular  variables  as  SOS1  variables  as  in 
formulation  (4.5). 

Step  4:  Solve  the  problem  again  using  the  solution  from  Step  1  as  an  initial 
starting  point. 

Step  5:  Go  to  Step  2. 

Step  6:  Check  solution  by  changing  value  of  L  in  fonnulation.  Decrease  L 
until  value  for  objective  function  stays  the  same.  Then  stop. 


4.2.5.  Numerical  Results 

Consider  the  following  sample  MPEC  where  three  finns  compete  to  sell  natural  gas 
in  the  market.  Assume  linear  demand  and  a  quadratic  cost  function.  This  MPEC  is 
modeled  as  a  Stackelberg  game  (Gibbons,  1996),  where  the  firms  choose  quantities  to 
produce.  In  this  context,  a  Stackelberg  game  is  relevant  under  the  assumption  that  a 
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shale  gas  producing  firm  can  exert  market  power  in  the  North  American  natural  gas 
market.  This  assumption  can  be  interpreted  in  various  ways.  One  way  is  that  other 
players  wait  for  the  shale  producing  firm  to  make  its  production  decision  before 
deciding  on  their  own  production  values.  Another  interpretation  is  that  the  shale 
producing  firm  can  influence  market  dynamics  so  that  the  other  players  become 
reactionary.  The  Stackelberg  leader,  “Shale  Firm,”  has  market  power  and  gets  to 
move  first  while  the  other  two  firms  are  followers. 
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Table  4.1:  Definition  of  terms  for  simple  example 


Parameters 

Shale  Firm 

Finn  1 

Firm  2 

Intercept  and  Slope  of  Linear  Demand 

a,b 

a,b 

a,b 

Marginal  cost 

C 

Cl 

C2 

Positive  Constants  Used  to  Replace 

N/A 

K, 

k2 

Complementarities  by  Disjunctive  Constraints 

Variables 

Shale  Finn 

Finn  1 

Firm  2 

Quantity  Natural  Gas  Sold"16 

0 

qi 

Binary  Variables  Used  to  Replace 

N/A 

ri 

r2 

Complementarities  by  Disjunctive  Constraints 

Outputs 

Shale  Firm 

Finn  1 

Firm  2 

Market  Price” ' 

P 

P 

P 

Profits 

ProfitShale 

Profitl 

Profit 2 

Shale  Firm  solves  a  constrained  maximization  problem  where  it  maximizes  its 
own  profits.  This  is  the  upper-level  problem: 

max  {(a  - b(qx  +q2+  Q))Q  -  CQ }  (4.9) 


26  These  quantities  are  constrained  to  be  nonnegative. 

27  We  assume  a  linear  demand  with  P  =  a  -  b(q\  +  q2  +  Q). 
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The  Anns  i  =  1,2  at  the  lower-level  solves  the  following  problem  where  they 
take  quantity  Q  as  given  and  try  to  maximize  profits  while  in  Nash-Cournot 
competition  with  the  other  Stackelberg  follower  firm  j. 

max  {(a - b(qi  +q.+  Q))qi  - ciqi }  (4.10) 

qt  >0 

This  lower-level  Nash-Cournot  game  can  be  expressed  as  a  (linear) 
complementarity  problem  given  as  follows: 


0  <  -a  +  cx  +  2bqi  +  bq2  +  bQ  _L  qx  >  0 
0  <-a  +  c2  +  2bq2  +  bql  +  bQ  -Lq2>0 


(4.11) 


To  solve  the  problem  using  disjunctive  constraints,  the  KKT  conditions  are 
added  to  the  constraint  set  in  (4.9)  to  fonn  one  overall  problem.  By  having 
sufficiently  large  positive  constants  K\  and  Ki,  the  complementarity  problem  (4. 1 1)  is 
refonnulated  as  follows: 


0  <  -a  +  Cj  +  2bqi  +  bq2  +  bQ  <  K{r\ 
0<4i  <Kx(\-rx) 

0<-a  +  c2+2bq2+bql+bQ  <  K2r2 
0<q2  <  K2(l-r2) 


(4.12) 


where  r\  are  ri  are  binary  variables.  Let  K\  =  be  the  maximum  of  the  x-intercept,  y- 
intercept  of  the  demand  function,  and  the  capacity  restrictions,  i.e.  K\  =  Ki_  =  max 
{a/b,  a } .  This  provided  a  lower  bound  on  K\  and  Ki  so  that  there  isn"t  a  computational 
error  (Gabriel  &  Leuthold,  2010). 

Finally,  replacing  the  original  complementarity  problem  with  the  disjunctive 
constraints  and  combining  with  the  upper-level  problem,  the  following  mixed-integer 
nonlinear  program  fonnulation  is  expressed  in  disjunctive  fonn: 
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max  {(«  -  b{<! i  +(h+  Q))Q  ~  CQ} 

Q'h  .92 

0  <  —a  +  c1  +  2 bq{  +  bq2  +  bQ  <  Kq\ 

0  <Qi  <^(1-^) 

0  <  —a  +  c2  +  2 bq2  +  bqx  +  bQ  <  K2r2 
0<q2  <K2(l-r2) 
r\,r 2  e{0,l} 


The  goal  is  to  use  (4.13)  as  a  benchmark  for  comparison  to  the  proposed 
method.  Using  (4.9)  to  (4.11),  the  MPEG  under  consideration  is  reformulated  to 
demonstrate  the  SOS1  and  penalty  methods. 


Zj  =  -a  +  q  +  2bqx  +  bq2  +  bQ 
z2  =-a  +  c2+  lbq2  +  bqx  +  bQ 

Zj  >  0 

z2  >0 


(4.14) 


Zitfi  =  0 
z2q2  =  0 


Now,  for  i  =  1,  2,  ziqi  =  u:  -  v(.  where 


u  =  +  Z‘  and  v,  =  — — —  by  Schur"s 

'  2  '  2 


decomposition.  So  the  eventual  formulation  using  SOS  type  1  variables  is: 
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max  \{a  -  b(qx  +  q2  +  (Q))(Q  -  CQ } 

Q<q i,q2 

Zj  =  -a  +  Cj  +  2bqx  +  bq2  +  bQ 
z2=-a  +  c2  +  lbq2  +  bqx  +  bQ 

Zj  >  0 

z2>0 

_  qx  +  z, 

pii  — 

1  ^ 


Vj  -Vj  = 


qx-zx 


q2  +  z2 


ux  -  (Vj+  +  vx  )  =  0 
m2-(v2+  +  v2)  =  0 

where  v;+ ,  v;  are  SOS  typel  variables 


(4.15) 


Similarly,  the  formulation  for  the  penalty  method  is  given  by 
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max 

2 


(a  -  h(q]  +q2  +  Q))Q  -  CQ-L (£  v,+  +  v“) 


i=i 


Zj  =  -a  +  q  +  2bqx  +  bq2  +  bQ 
z2=-a  +  c2  +  lbq2  +  bqx  +  bQ 

Zj  >  0 


z2  >0 


U\  = 


ql  +  zl 


q\  —  z\ 


q2  +  z2 


q2  z2 


(4.16) 


ux  -  (Vj+  +  Vj  )  =  0 
m2-(v2+  +  v~)  =  0 

where  v(+ ,  vT  are  non  -  negative  variables 


The  computational  objective  was  to  compare  the  results  for  the  methods  of 
disjunctive  constraints  (4.13),  SOS1  variables  (4.15)  and  the  penalty  method  (4.16). 
This  was  accomplished  using  GAMS  (GAMS,  2010)  with  CONOPT  being  used  as 
the  nonlinear  solver  and  SBB  the  mixed  integer  nonlinear  solver. 

The  capacity  constraints  for  production  quantities  are  omitted  for  this  first 
example,  i.e.,  maximum  production  is  only  limited  by  rising  costs.  The  following 
Table  4.2  shows  the  datasets  used: 
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Table  4.2:  Different  Datasets  to  Compare  (4.13),  (4.15),  and  (4.16) 


Dataset  Parameters 

Dataset  1 

Dataset  2 

Dataset  3 

a 

13 

13 

13 

b 

1 

0.1 

0.1 

cj  =  c2  =  C 

1 

1 

2 

For  the  disjunctive  constraints  fonnulation  (4.16),  K\  =  Ki  =  max  {b/a  ,  a}  = 
13  consistent  with  (Gabriel  &  Leuthold,  2010)  for  all  the  datasets  while  for  the 
penalty  method  approximation  (4.16),  two  different  values  of  L  were  chosen  to  show 
how  a  lower  value  of  L  gives  a  better  answer  as  shown  below  in  Table  4.3. 


Table  4.3:  Different  Cases  to  Compare  Solutions  to  (4.16) 


Case  1 

Case  2 

Value  ofZ 

0.0001 

1 

28 

The  following  Tables  4. 4-4. 6  report  the  results.  The  true  answer  can  be 
easily  verified  algebraically  as  unique  and  is  shown  in  the  third  column  of  the  tables. 
Note  that  disjunctive  constraints  obtained  the  correct  answer  for  Dataset  1,  implying 
that  a  correct  value  of  K  was  chosen. 


28  It  is  simple  algebra  to  show  that  this  is  the  unique  solution  since  there  are  no  constraints  and  all 
objective  functions  are  quadratic. 
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Table  4.4:  Results  for  Dataset  1 


Results 

Disj  Cons 

True 

Answer 

SOS 

Casel 

Case  2 

qi  =  q2 

2.000 

2.000 

2.000 

2.000 

1.833 

Q 

6.000 

6.000 

6.000 

6.000 

6.500 

Price 

3.000 

3.000 

3.000 

3.000 

2.833 

Profit  shale 

12.000 

12.000 

12.000 

12.000 

11.917 

Profit  1=2 

4.000 

4.000 

4.000 

4.000 

3.361 

Table  4.5:  Results  for  Dataset  2 


Results 

Disj  Cons 

True 

Answer 

SOS 

Casel 

Case  2 

<N 

II 

S' 

13.000 

20.000 

20.000 

20.000 

18.333 

Q 

81.000 

60.000 

60.000 

60.000 

65.000 

Price 

2.300 

3.000 

3.000 

3.000 

2.833 

Profit  shale 

105.300 

120.000 

120.000 

120.000 

119.167 

Profit  1=2 

16.900 

40.000 

40.000 

40.000 

33.611 

Table  4.6:  Results  for  Dataset  3 


Results 

Disj  Cons 

True 

Answer 

SOS 

Casel 

Case  2 

qi  =  q2 

13.000 

18.333 

18.333 

18.333 

16.667 

Q 

71.000 

55.000 

55.000 

55.000 

60.000 

Price 

3.300 

3.833 

3.833 

3.833 

3.667 

Profit  shale 

92.300 

100.833 

100.833 

100.833 

100.00 

Profit  1=2 

16.900 

33.611 

33.611 

33.611 

27.778 

If  the  methodology  to  choose  K  as  outlined  in  the  literature  (Gabriel  & 
Leuthold,  2010)  is  used,  disjunctive  constraints  do  not  provide  the  solutions  in 
datasets  2  and  3“  .  These  results  point  out  a  big  weakness  with  disjunctive  constraints 
that  the  solution  can  be  very  far  from  the  true  answer  and  the  given  solution  can  be 
extremely  sensitive  to  the  value  of  K  if  appropriate  problem  specific  values  are  not 
selected. 

Choosing  the  correct  K  can  make  the  disjunctive  constraint  method  (4.13) 
accurate.  Choosing  a  correct  L  makes  (4.16)  accurate  as  well.  The  next  set  of 
numerical  results  were  done  with  Dataset  3  with  K  =10000  and30  L  =  10’16  where 
these  values  were  reached  after  numerical  and  algebraic  verification  of  the  test 
problem.  The  test  problem  was  changed  so  that  now  instead  of  two  players  at  the 

29  The  method  in  (Gabriel  &  Leuthold,  2010)  gives  a  correct  value  of  K  whenever  maximum 
production  (capacity  constraints  for  production)  is  included  in  the  problem  formulation.  Our  goal  was 
to  give  a  very  simple  counterexample  where  the  disjunctive  constraints  approach  didn't  work. 

30  This  is  machine -e. 
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lower  level,  there  were  M  players  with  similar  costs  and  parameters.  The  number  of 
players  was  increased  to  test  the  computation  time  taken  for  disjunctive  constraints 
(4.13),  SOS1  (4.15),  and  the  penalty  method  (4.16).  The  results  are  shown  in  the 
following  Figure  4.1.  All  methods  were  able  to  obtain  the  correct  solutions. 


Computational  Time  to  Solve  for  Number  of 

Players 


. SOS  1 

-  -Penalty 

- DisjunctiveConstraints 


Figure  4.1:  Computational  Time  for  Solving  Problem 

Clearly,  the  disjunctive  constraint  method  becomes  extremely  computationally 
expensive  when  number  of  players  is  increased.  Note  that  the  graphs  for  the  penalty 
and  SOS  1  methods  are  overlapping. 


Ill 


4.3.  Solving  Equilibrium  Programs  with  Equilibrium  Constraints 


4.3.1.  Extending  Algorithm  4.1  to  Equilibrium  Programs  with 
Equilibrium  Constraints 

Note  that  a  variation  of  the  above  fonnulation  (4.6)  can  also  be  used  to  solve  EPECs. 

An  EPEC  is  defined  as  a  game  between  N  players  at  the  top  level  where  each  top- 

level  player  solves  an  optimization  problem  of  the  form  of  an  MPEC.  Hence,  an 

EPEC  with  a  common  lower-level  for  each  of  the  N  upper-level  players  typical  of 

Stackelberg  leaders  in  energy  production  with  the  rest  of  the  market  represented  by 

the  lower-level  problem  is  given  by 

min  fj(x,y)  y=l,...,A 
s.t.(x,y)  eQ 

k>0  (4.17) 

yTg(x,y)  =  0 

The  formulation  (4. 17)  with  Q  =  {(x,  y)  \  C(x,  y)  <  0}  can  be  rewritten  as 

n 

min  fj(x,y)  +  LYj(vi+  +  v.~)  j  =l-..,N 

i= 1 


C(x,y)<  0 

w 

-T<0 

(T2) 

~g(x,y)<  0 

w 

^+vl.y+g^yK  o 

2 

w 

(y^v-y-y-z^Ko 

2 

where  v+ ,  v  are  non  -  negative  variables 


By  Theorem  4.2,  choosing  a  positive  L  will  ensure  that  the  SOS1  constraints 
hold  for  each  pair  v  V  for  each  individual  top-level  player"s  optimization  problem 
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and  choosing  a  small  enough  L  will  ensure  that  the  correct  solution  is  achieved. 
Formulation  (4.18)  can  then  be  solved  as  a  Nash  game  among  N  players,  and  can  be 
formulated  as  a  complementarity  problem  by  taking  the  Karush-Kuhn-Tucker 
conditions  as  in  (4.19). 

o  ^  VJj  (x,  y )  +  4,' v xC(x,  y )  -  A}  V Xg(x,  y)  -  y  Vxg(x,  y)  +  y  Vxg(x,  y)  1  x  >  0 

0  ^  Vyfj  {x,y)  +  K V  yC(x,  y)  -  A3Vyg(x,  y)  -  y  (1  +  V  yg(x,  y))  -  y  (1  -  Vvg(x,  y))  T  y  >  0 

0  <  ivf  +  44v,+  +  Ajvf  _L  v,+  >  0 
0  <  Lvf  +  A4V/~  -  AfVj-  _L  vf  >  0 

*  =  l,...,Vy  =  l,...,jV  (4.19) 

0<-C(wy)-L4!  >0 
0<g(x,y)±Jj  >0 


(v‘+v-)  + 

2 

(A 

free) 

2 

a5 

free) 

Note  that  as  described  in  detail  in  (Ehrenmann,  2004),  (4.19)  is  not  a  square 
system.  The  reason  for  this  is  that  the  same  set  of  lower-level  variables  are  shared 
among  all  top-level  players  in  the  component  MPECs  of  the  EPEC.  Hence,  solutions 
to  (4.19)  cannot  be  computed  using  solvers  in  GAMS.  Many  workarounds  are 
available  for  this  (e.g.  penalization  methods  (Ehrenmann,  2004)),  and  for  the  case  of 
the  specific  EPEC  considered  in  this  chapter,  we  introduce  a  “balancing  agent.”  All 
lower-level  variables  and  constraints  that  are  common  to  all  top-level  player 
optimization  problems  are  treated  as  separate  variables  for  each  top-level  player.  For 
example,  the  variables  y  above  will  be  treated  as  separate  variables  V/  for  each  of  the 
top-level  player  MPECs.  Then,  (4.19)  combined  with  the  Karush-Kuhn-Tucker 
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-5  1 

conditions  of  (4.20)  below  will  be  a  square  system.  The  balancing  agent  solves  the 
following  problem 


N- 1 


min 

bJJ>j  M 


^+b: 


yj~yj+i-bj+b]=o 

b;,b:>0 
j  =  l,...,N-l 


O/) 


(4.20) 


A  loose  interpretation  of  the  economic  role  of  the  balancing  agent  is  the 
following.  Without  such  an  agent,  Stackelberg  leader  j  communicates  with  only  the 
/th  partition  of  the  lower-level  market,  represented  by  yr  The  balancing  agent  tries  at 
minimal  cost  to  couple  the  partitions  into  one  integrated  market,  which  is  more 
realistic.  The  Karush-Kuhn-Tucker  conditions  of  the  above  problem  can  then  be 
added  to  (4.19)  to  ensure  that  the  values  of  values  of  v,  are  the  same.  The  following 
algorithm  shows  a  method,  then,  to  solve  EPECs. 


4.3.2.  Algorithm  4.2  to  Solve  Equilibrium  Problems  with  Equilibrium 
Constraints  (Heuristic) 

(Solving  EPECs  Using  (4.19)  and  Karush-Kuhn-Tucker  conditions  of  (4.20)) 


Step  0:  Pick  a  tolerance  t. 


31  Since  (4.20)  is  a  linear  program,  the  Karush-Kuhn-Tucker  conditions  are  both  necessary  and 
sufficient. 
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Step  1:  Solve  the  problem  using  the  fonnulation  (4.19)  with  L  =t  and  the  KKT 
conditions  to  (4.20). 

Step  2:  Check  solution  by  changing  value  of  L  in  formulation.  Decrease  L  until 
value  for  objective  function  stays  the  same.  Check  with  setting  L  = 
machine -e.  Then  stop. 


4.3.3.  Numerical  Results  for  Equilibrium  Programs  with  Equilibrium 
Constraints 

A  corresponding  EPEC  where  two  players  are  at  the  top-level  can  also  be  formulated 
and  solved  by  extending  the  MPEC  method  above.  The  fonnulation  for  the  bottom 
level  remains  the  same,  and  for  the  upper  level,  there  are  now  two  producers  who 
detennine  quantities  Q\  and  Qi  whose  objective  functions  are  given  as 

{(a-h(ql  +  q2  +  Ql+Q2))Qj-CjQj}  j  =  1,2  (4.21) 

32 

Using  the  same  datasets,  let  C\  =  Ci  =  C.  Then,  the  EPEC  can  be  fonnulated 
as 


32  This  also  works  with  other  data,  which  was  verified  numerically  as  well. 
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max  (a  - b(q1  +  q2+Qx+Q2 ))Q  -  C  Qj  -  L(J^  v,+  +  vf)  \  j  =  1,2 

Qj  .91  .ft  (  ' 

-a+cx  +  2bqx  +  bq2  +  bQ ,+ bQ 2>  0 
-a  +  c2  +2bq2  +  bqx  +  bQ  x+  bQ  2>  0 

ql+(~a  +  cl+2bql+bq2+bQl+bQ2) 

V,  “t-  Vi 

1  1  2 

+  _  qx  -{-a  +  q  +2bql  +bq2  +bQ1+bQ2) 


q2  +(-a+c2  +2 bq2  +  bqx  +  bQx+bQ2)  _ 
2 

q2  -{-a  +  c2  +2 bq2  +  bqx  +bQl+bQ2 )  _ 
2 


v,  -v,  - 


W 

(A) 

=  0 

(A) 

=  0 

(A) 

=  0 

(A) 

=  0 

(A) 

(4.22) 


where  v / ,  vi  are  non  -  negative  variables 

The  constraints  in  (4.22)  are  the  KKT  conditions  of  the  lower-level  problem 


that  have  been  reformulated  as  in  (4.6).  As  described  in  Section  4.3.1,  this  problem 


can  be  expressed  and  solved  as  a  complementarity  problem  using  Algorithm  4.2  and 


adding  a  balancing  agent: 
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X,  ,b  X.  b  X*  b  X,  h 

0  <-a  +  Cj+  bqt  +bq2+bQl+bQ2+bQJ--±j-  +  -± - 1-  +  1  Q  >  0 


0  <  b(Q  -  2bXl  j  -  bX2J  +  X3  J 


0  <  bQ-  bXlJ  -  2 bX2  j  +  X5J 


1  +  2^ 


2  j+A*’j 


b 


1  +  2  b 


+  Aj  ~ 


1-2  b 


f  b 


-b 

2 

-b 


—  +  X,  i - ±  q ,  >  0 

2  I  6j\  2  lu 


+  2)  +  ^\~^fr\Lq V"  0 


0  <  I  +  X2MJ  +  X2i+2j  1  vlj  >  0 


0  <  I  +  X2MJ  -  X2i+2J  1  v~j  >  0 
0  <  -a  +  q  +  2bql +  bq2J  +  bQx+  bQ2L  XtJ  >  0 
0  <-a  +  c2  +  2 bq2  j  +  bqx ;  +  bQ ,+  bQ 2_L  X2  j  >  0 

qXj  +  (-a  +  cx  +  2  bqXj  +  bq2j  +bQl+bQ2) 


VU+VU  — 


vi  j  ~frj  - 


V2„,  +  V2,,  - 


V2,i  ~  V2,J  - 


qtJ  ~  (- a  +  Cj  +  2 bqUj  +  bq2  j  +  bQx+bQ2) 
2 

q2,j  +  (-«  +  c,  +  2bq2j  +  bqXJ  +  bQl+bQ2) 


<hj  ~  ( ~a  +  c2  +  2-bq2  j  +  bqX  j  +bQl+bQ2) 


=  0 

(4, 

free) 

=  0 

(Kj 

free) 

=  0 

(A,; 

free) 

=  0 

(A,,- 

free) 

(4.23) 


0  <  1  -  ^  ±6+  >0 
0  <  1  +  7Tj  ±  fr  >  0 

9ij  ~  Vi, 3 -j  ~  hj  +  bj  =  0  (rjj  free) 

bfbj>0 

j  =  1,2,  i=l,2 

The  following  tables  (4. 7-4. 9)  give  the  solutions  under  different  datasets  and 


cases  .  Simple  algebra  can  show  that  there  exists  a  solution,  and  hence  a  true  answer 
is  also  given  in  the  table. 


33  Many  different  cases  with  different  costs  were  also  solved  successfully,  but  only  the  ones 
corresponding  to  the  previous  MPEC  example  are  presented.  Please  refer  to  Section  4.2.5. 
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Table  4.7:  Results  for  Dataset  1 


Results 

A  Solution 

Case  1 

Case  2 

L  =  in  a  chi  lie-:: 

>-*» 

II 

i-O 

Kj 

1.333 

1.333 

1.111 

1.333 

Qi  -  Q2 

4.000 

4.000 

4.333 

4.000 

Price 

2.333 

2.333 

2.111 

2.333 

Profit  Top 

5.333 

5.333 

4.815 

5.333 

Profit 

Bottom 

1.778 

1.778 

1.235 

1.778 

Table  4.8:  Results  for  Dataset  2 


Results 

A  Solution 

Case  1 

Case  2 

L  =  machine-:: 

qi  =  q2 

13.333 

13.333 

11.111 

13.333 

Qi  =  Qi 

40.000 

40.000 

43.333 

40.000 

Price 

2.333 

2.333 

2.111 

2.333 

Profit  Top '4 

53.333 

53.333 

48.148 

53.333 

Profit  Bottom 

17.778 

17.778 

12.346 

17.778 

Table  4.9:  Results  for  Dataset  3 


Results 

A  Solution 

Case  1 

Case  2 

L  =  machine-^ 

qi  =  qi 

12.222 

12.222 

10.000 

12.222 

Qi  =  Q2 

36.667 

36.667 

40.000 

36.667 

Price 

3.222 

3.222 

3.000 

3.222 

Profit  Top 

44.815 

44.815 

40.000 

44.815 

Profit  Bottom 

14.938 

14.938 

10.000 

14.938 

Again,  in  all  three  datasets,  the  Case  2  choice  of  L  could  not  give  an  optimal 
solution,  i.e.,  Nash  equilibria  at  the  top  and  bottom.  Hence,  all  these  datasets  required 
a  very  small  choice  of  L.  Choosing  L  =  machine-e  is  a  good  option.  Note  that  the 
EPEC  has  an  extra  player  when  compared  to  the  MPEC;  hence  profits  for  all  firms 
are  lower  in  the  EPEC  study.  Moreover,  prices  are  also  lower  in  the  EPEC  case,  as 


34  Due  to  the  deliberate  selection  of  similar  data,  the  profits  for  both  top-level  players  are  the  same. 
Hence,  only  one  player"s  profit  is  reported  as  the  second  player"s  profits  are  the  same. 
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expected  with  an  extra  Stackelberg  leader  with  same  marginal  cost  in  the  EPEC. 
These  results  can  be  seen  when  comparing  Tables  (4.4-4. 6)  with  Tables  (4. 7-4. 9). 

4.4.  The  North  American  Gas  Model 

4.4.1.  Introduction 

The  advent  of  rising  oil  prices  along  with  attitudes  about  decreasing  greenhouse  gas 
emissions  in  multiple  sectors  has  lead  to  an  interest  in  natural  gas  production  for  the 
future.  The  role  of  unconventional  gas  ,  in  particular,  has  greatly  increased  due  to 
engineering  advances  such  as  hydraulic  fracturing  and  horizontal  drilling  (NPC, 
2007).  The  projected  role  of  shale  gas  in  particular,  especially  in  the  United  States  but 
also  elsewhere  (Skagen,  2010)  has  lately  been  a  major  force  in  the  increasing 
prominence  of  unconventional  gas.  In  2008,  Cambridge  Energy  Research  Associates 
indicated  that  this  unconventional  gas  production  could  help  delay  by  a  decade  the 
United  States"  need  for  substantial  LNG  imports  (Economist,  2008).  More  recently, 
others  gauge  the  U.S.  shale  gas  impact  in  even  more  dramatic  terms  with  estimates  of 
up  to  100  years  of  reserves.  Indeed,  the  Potential  Gas  Committee  has  concluded 
that  the  U.S.  proved  reserves  of  gas  increased  from  2006  to  2008  by  a  huge  35.4% 
from  1532.0  trillion  cubic  feet  to  2074.1  (PGC,  2010).  Others  such  as  the  petroleum 


35  Unconventional  gas  is  defined  as  gas  from  tight  sands,  coalbed  methane,  and  shale  gas,  and  covers 
more  low-permeability  reservoirs  that  produce  mostly  natural  gas  (no  associated  hydrocarbon  liquids) 
(NPC,  2007). 

36  Keith  O.  Rattie,  CEO  of  Questar,  a  natural  gas  and  pipeline  company,  cited  in  “Awash  in  Fossil 
Fuels,”  George  F.  Will,  The  Washington  Post,  November  22,  2009. 
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engineer  Art  Berman  are  more  cautious  about  the  ultimate  supply  due  to  the 
economics  of  producing  shale  gas  (low  gas  prices  in  the  U.S.  recently)  (Cohen,  2009) 
or  steeper  decline  rates  for  shale  wells  (Steffy,  2009).  Shale  gas  in  the  U.S.  will  be 
modeled  using  the  World  Gas  Model  (Gabriel  et  ah,  2011c),  restricted  to  North 
American  nodes. 

'in 

The  World  Gas  Model  (WGM)  is  a  long-tenn,  game  theoretic  model  of 
global  gas  markets  with  representation  of  Cournot  market  power  originally  based  on  a 
North  American  version  of  the  mode  (Gabriel  et  ah,  2005a),  (Gabriel  et  ah,  2005b) 
and  eventually  extended  to  a  global  version  (Egging  et  ah,  2009)  for  which  the  most 
recent  version  is  (Gabriel  et  ah,  2011c).  For  the  United  States,  the  forecasts  presented 
in  the  Annual  Energy  Outlook  (April  2009  ARRA  version)  were  used  for  the  current 
study.  For  the  rest  of  North  America,  the  World  Energy  Outlook  (IEA,  2008)  was 
used.  The  WGM  was  then  extensively  calibrated  to  match  these  multiple  sources  for 
all  countries/aggregated  countries  and  years  considered  (2005,  2010,  2015,  2020, 
2025,  2030). 38 

The  most  interesting  change  due  to  the  presence  of  shale  gas  occurs  in  census 
region  7  (WGM  node  7)  where  Haynesville  and  Barnett  plays  are  present.  This  node 


37  National  Science  Foundation  (DMS,  Award  #0106880),  Principal  Investigator  ,S.A.  Gabriel, 
“Computational  Methods  for  Equilibrium  Problems  with  Micro-Level  Data,”  09/01/2001-08/31/2005 
and  National  Science  Foundation  (DMS,  Award  #0408943),  Principal  Investigator,  S.A.  Gabriel, 
“Methods  and  Models  for  Stochastic  Energy  Market  Equilibria,  “08/01/2004-07/31/2008. 

38  See  (Gabriel  et  al.,  2011c)  for  details  on  the  countries  and  regions  included  as  well  as  other  relevant 
geographic  or  nodal  information. 
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is  used  as  a  test  example  for  the  new  MPEC  solution  technique.  The  fonnulation 
proposed  is  that  the  producer  of  shale  at  node  7  will  be  the  first  mover  in  the 
Stackelberg  game  (Gibbons,  1996).  The  entire  lower  level  will  be  the  World  Gas 
Model  restricted  to  North  American  nodes  including  other  shale  nodes.  This 
formulation  assumes  that  the  shale  producer  at  node  7  has  market  power  over  all  other 
players.  While  there  is  some  arbitrariness  about  this  assumption,  i.e.  it  might  give  the 
shale  producer  at  node  7  too  much  market  power,  it  nevertheless  is  an  interesting 
market  fonnulation  to  study  because  there  is  a  chance  for  this  scenario  to  play  out  in 
the  future.  Furthennore,  this  fonnulation  can  be  used  for  bounding  purposes  when 
considering  a  wide  variety  of  market  dynamics.  Other  MPEC  formulations  might 
consider  a  trader,  producer,  or  even  the  government  at  the  upper-level. 

4.4.2,  Shale  Gas  in  the  United  States 

The  shale  gas  data  were  provided  by  the  U.S.  Department  of  Energy  in  the 
Annual  Energy  Outlook  (2010)  with  shale  gas  production  and  Lower  48  onshore 
natural  gas  production  datasets.  As  compared  to  the  version  of  the  model  from 
(Gabriel  et  ah,  2011c),  the  World  Gas  Model  was  modified  to  contain  three 
production  nodes  for  each  census  region  of  the  United  States:  conventional  gas,  shale 
gas,  and  non-shale  unconventional  gas. 

A  „Golombek"  production  cost  function  (Golombek  &  Gjelsvik,  1995) 

c(?)=  (a  -  r)q  +  fiq1  +  r(Q  ~  q)  Inf  (4.28) 

39  For  a  table  relating  the  WGM  nodes  to  the  shale  plays  in  the  US,  please  refer  to  the  Appendix. 
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was  used  for  which  the  marginal  supply  cost  curve  is: 


C(«)  -a  +  Pq  +  y  In 


Q-q ' 

e ; 


(4.29) 


Here,  0  is  the  production  capacity,  a  >  0  is  the  minimum  per  unit  cost,  /?  >  0 
is  the  per  unit  linearly  increasing  cost  tenn,  and  /  <  0  is  a  term  that  induces  high 
marginal  costs  when  production  is  close  to  full  capacity. 

Skagen  (Skagen,  2010)  indicates  that  recent  research  has  led  to  predicting  a 
lower  value  of  a  for  the  cost  function  of  shale  gas  when  compared  to  conventional 
gas.  Figure  4.2  shows  that  shale  gas  is  now  understood  to  have  a  lower  price  of 
extraction  in  the  beginning. 


Prior  Perception  New  Understandin 


Gas  Resources  Gas  Resources 

Figure  4.2:  A  Marginal  Cost  Structure  for  Shale  Gas  (Skagen,  2010) 


Alternatively,  others  believe  that  initial  positive  results  from  shale  gas 
extraction  wells  might  not  be  sustainable  in  the  long  run  (Cohen,  2009).  In  particular, 
geologist  Art  Berman  claims  that  decline  rates  will  be  much  higher  than  expected, 
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and  while  shale  appears  to  be  a  good  resource  right  now,  steep  decline  rates  mean  that 
higher  extraction  will  lead  to  higher  costs  quickly  (Cohen,  2009). 

In  the  modification  of  the  WGM,  the  shale  gas  cost  curve  has  a  (the  y- 
intercept  of  the  marginal  cost  curve)  lower  and  />'  (the  slope  of  the  marginal  cost 
curve)  higher  than  for  conventional  gas.  The  current  debate  about  shale  gas  has  been 
incorporated.  While  the  lower  initial  cost  of  extraction  is  consistent  with  Skagen"s 
observation,  a  higher  marginal  cost  increase  and  higher  marginal  costs  at  higher 
quantities  is  consistent  with  BermarTs  claim  that  decline  rates  of  shale  wells  will  be 
higher.  Hence,  shale  gas  has  a  lower  initial  cost  of  extraction  than  conventional  gas 
but  a  higher  rate  of  increase  for  marginal  cost.  It  is  important  to  note  that  this 
marginal  cost  curve  for  shale  gas  is  by  no  means  the  final  word  but  just  one 
perspective  developed  for  our  modeling  needs. 

The  other  initial  condition  placed  is  that  total  production  costs  should  be  the 
same,  so  the  integral  of  the  marginal  cost  curve  should  be  the  same  for  both  functions 
(conventional  and  shale  gas).  This  will  ensure  a  positive  production  of  both  types  of 
gas,  which  can  be  calibrated  to  real  data.  A  comparison  is  provided  with  two  other 
cases  with  higher  total  costs  for  shale  production.  Another  reason  why  the  total  costs 
would  be  equal  in  the  reference  case  is  that  producers  drilling  in  the  same  region 
would  encounter  similar  terrain,  similar  taxes,  similar  hurdles  etc.  Hence,  a  was 
reduced  by  20%  of  the  value  of  conventional  gas  based  on  Skagen  (2010)  and  /?  was 
increased  by  an  amount  so  that  the  integral  of  the  marginal  cost  curve  remains  the 
same.  An  explanation  of  this  is  shown  in  Figure  4.3  below.  Note  that  the  values  of  y 
are  kept  the  same  for  shale  and  conventional  gas,  so  Figure  4.3  only  shows  the  linear 


124 


portion  of  the  marginal  cost  curve.  The  production  cost  data  for  conventional  and 
unconventional  (non-shale)  gas  was  obtained  as  described  in  (Gabriel  et  ah,  2011c). 


A  Marginal  Cost  Curve  for  Shale  Gas  Production 


Shale  Gas  Marginal  Cost  Curve 


o 

U 

•§)i 


Shale  Gas  Marginal 
Cost  curve  has  20% 
lower  intercept 


Shale  Gas 
Marginal  Cost 
curve  has 
steeper  slope 


Shale  Gas  Marginal  Cost  curve  slope  is 
obtained  by  setting  the  total  area  under  the 
curves  (total  cost)  equal  to  each  other. 


Quantity  of  Gas  Produced 

Figure  4.3:  A  Marginal  Cost  Structure  for  Shale  Gas 


The  following  table  provides  the  coverage  of  states  and  shale  basins  in  the 
world  gas  model. 
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Table  4.10:  World  Gas  Model  Nodes:  Coverage  of  States  and  Shale  Basins 


Shale  Basin 
Name 

States 

WGM 

Nodes 

Mancos 

Utah 

US_8 

Hilliard- 

Baxter 

Mancos 

Wyoming 

Colorado 

US_8 

Niobrara 

Colorado 

Nebraska 

Kansas 

US  4, 

US  8 

Cody 

Montana 

US_8 

Mowry 

Wyoming 

US_8 

Gammon 

Montana 

North 

Dakota 

South 

Dakota 

US  4, 

US  8 

Excello- 

Mulky 

Kansas 

Oklahoma 

US  4, 

US  7 

New  Albany 

Illinois 

Indiana 

Kentucky 

US  3, 
US_6 

Antrim 

Michigan 

Indiana 

Ohio 

US_3 

Utica 

New  York 

US_2 

Marcellus 

New  York 

Pennsylvania 

Ohio 

West 

Virginia 

Maryland 

Virginia 

Tennessee 

US  2, 

US  3, 

US  5, 

US  6 

Devonian 

Ohio 

Kentucky 

West 

Virginia 

Virginia 

Tennessee 

Alabama 

Georgia 

US  3, 

US  5, 
US_6 

Chattanooga 

Kentucky 

Virginia 

Tennessee 

Alabama 

Georgia 

US  5, 

US  6 

Conasauga 

Alabama 

Georgia 

US  5, 
US_6 

Floyd-Neal 

Mississippi 

Alabama 

US_6 

Fayetteville 

Arkansas 

US_7 

Hayneville/ 

Bossier 

Louisiana 

Texas 

US_7 

Woodford/ 

Caney 

Oklahoma 

US_7 

Barnett 

Texas 

US_7 

Pearsall 

Texas 

US_7 

Woodford 

Oklahoma 

Texas 

US_7 

Barnett  and 
Woodford 

New  Mexico 

Texas 

US  7, 

US  8 

Bend 

Texas 

US_7 

Pierre 

New  Mexico 

Colorado 

US_8 

Lewis 

New  Mexico 

Colorado 

US_8 

Hermosa 

Utah 

US_8 

4.4.3.  Scenario  Results 

This  section  describes  numerical  examples  to  solve  the  WGM  (North  American  nodes 
only)  using  the  new  MPEC  approach  outlined  above.  The  point  is  to  demonstrate  that 
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even  on  large  MPECs,  this  Algorithm  4.1  works  well.  Five  different  cases  were  run, 
which  are  described  below.  The  results  from  these  cases  are  consistent  with  economic 
theory,  and  are  presented  in  graphical  form. 

Computational  results  show  that  a  solution  exists  for  the  lower-level 
complementarity  problem  (which  includes  the  shale  producer  at  node  7).  This  means 
that  a  feasible  solution  for  the  MPEC  exists  as  well,  as  the  lower-level 
complementary  problem  contains  both  the  complementary  restrictions  as  well  as 
constraints  for  the  upper-level  player  of  the  MPEC.  The  method  of  disjunctive 
constraints  did  not  provide  a  feasible  solution  for  this  problem  with  the  solvers  SBB 
and  CONOPT  (GAMS,  2010). 

The  WGM  restricted  to  the  North  American  nodes  has  30  producers,  of  which 
seven  are  for  shale  gas  and  seven  for  unconventional  gas  production  in  the  United 
States.  The  rest  produce  conventional  gas.  There  are  a  total  of  15  production  nodes,  of 
which  nine  correspond  to  the  census  regions  for  the  lower-48  states.  There  are  also 
three  traders  (one  each  for  United  States,  Canada,  and  Mexico,  the  three  countries  in 
the  model),  along  with  eight  periods  from  2005-2040  (the  last  two  five-year  periods 
are  not  reported  to  avoid  end-of-horizon  bias),  and  two  seasons  (high  and  low 
demand)  in  each  period.  The  decision  variables  are  operating  levels  (production, 
storage  injection,  etc.)  as  well  as  investment  levels  (pipeline,  liquefaction  capacity, 
etc.).  Prices  are  set  to  2005  US$.  The  whole  complementarity  model  has  about  9456 
variables  and  takes  243.2  seconds  to  solve  on  a  2.0  GHz  processor  with  2  GB 
memory. 
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The  MPEC  version  of  the  WGM  restricted  to  North  America  was  formulated 


with  the  shale  gas  producer  in  census  region  7  as  the  top-level  player.  Census  region  7 
contains  both  the  Barnett  and  Haynesville  shale  plays,  two  of  the  most  important  ones 
in  the  United  States40.  The  MPEC  version  was  solved  using  Algorithm  4.1.  The 
algorithm  solved  the  problem  in  approximately  three  hours  on  the  same  computer 
described  above,  though  the  time  was  different  for  each  case. 

The  following  five  cases  were  considered,  with  the  first  (Base  Case)  modeled 
as  a  complementarity  problem  and  the  rest  as  MPECs  for  purposes  of  comparison: 

1)  Base:  The  Base  Case  for  the  WGM  restricted  to  North  America  fonnulated  as 
a  complementarity  problem  and  calibrated  according  to  the  Annual  Energy 
Outlook  (April  2009  ARRA  version)  and  the  World  Energy  Outlook  (IEA, 
2008). 

2)  MPEC:  The  MPEC  version  of  the  Base  Case.  The  shale  producer  in  census 
region  7  was  placed  at  the  upper  level  and  all  other  players  at  the  lower  level. 

3)  MoreShale:  A  higher  production  of  shale  gas  was  considered  by  increasing 
the  daily  capacity  available,  with  a  10%  increase  for  2015,  2020;  a  15% 
increase  for  2025,  2030;  and  a  20%  increase  for  2035,  2040.  These  numbers 
are  approximations  of  increases  given  by  the  Annual  Energy  Outlook  between 
the  2008  and  2009  reports"  predictions.  While  the  2010  reports  did  not  show 
such  an  increase,  for  our  purposes  this  case  was  developed  to  show  what 
might  happen  if  a  similar  increase  took  place  after  2015.  This  case  is  modeled 
as  an  MPEC. 


40  Refer  to  www.eia.gov  for  more  information. 
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4)  ShaleTax:  All  shale-producing  firms  are  taxed  $0.39/MCF  (39  cents  for  every 
thousand  cubic  feet  of  natural  gas  produced)  from  2015  to  2040.  This  is  in  line 
with  the  tax  proposed  for  Pennsylvania  shale  production  in  the  Marcellus 
shale  play,  which  was  later  overturned  (Bames,  2010).  No  other  value  for  a 
shale  tax  has  so  far  been  found  from  any  legislature.  This  case  is  modeled  as 
an  MPEC. 

5)  AllTax:  All  natural  gas  is  taxed  at  $0.39/MCF  from  2015  to  2040.  This  case 
will  help  see  if  the  shale  players,  especially  the  one  in  census  region  7,  have 
any  comparative  advantage  when  everyone  is  taxed.  Modeled  as  an  MPEC. 

The  results  are  presented  below.  The  MPEC  case  produces  lower  average 
prices  (e.g.,  $6.74/MMBTU  vs.  $6.94/MMBTU  in  2025)  and  higher  total  production 
(e.g.,  844.2  BCM  vs.  830.2  BCM  in  2025)  and  consumption  (Gibbons,  1996)  when 
compared  to  the  Base  Case  for  all  years.  Moreover,  as  expected,  the  MoreShale  case 
showed  an  overall  increase  in  shale  production  when  compared  to  the  Base  Case  (e.g., 
111.5  BCM  vs.  89.4  BCM  in  2025)  and  for  the  shale  producer  in  census  region  7, 
proved  to  be  the  most  profitable.  The  profits  at  node  7  increase  by  more  than  three 
times  in  2025  when  compared  to  the  Base  Case.  This  shows  the  advantage  of  being 
the  Stackelberg  leader  and  allowing  collection  of  more  profits  and  also  serves  as  a 
cautionary  numerical  result  for  market  regulators  and  other  interested  parties. 

The  MoreShale  case  shows  that  it  will  be  advantageous  for  producers  as  well 
as  consumers  (with  prices  dropping  in  nodes  with  large  amounts  of  shale).  However, 
the  fact  that  total  production  doesn"t  change  much  with  the  invoking  of  the  tax  (shale 
or  otherwise)  shows  that  it  will  not  be  detrimental  to  the  producers.  This  is 
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corroborated  by  looking  at  producer  profits  as  well,  where  the  imposition  of  tax 
barely  changes  total  profit.  Since  Node  8  has  a  relatively  abundant  supply  of 
conventional,  unconventional,  and  shale  gas,  it  can  change  production  around 
depending  on  the  demands.  Hence,  nodes  8  and  9  remain  relatively  unchanged  with 
the  imposition  of  tax.  Moreover,  the  production  for  shale  producers  is  as  expected, 
and  the  imposition  of  tax  does  less  to  harm  any  production,  and  overall  profits  remain 
relatively  stable.  Also,  this  might  be  a  policy  argument  for  saying  that  the  tax  will 
barely  harm  producers,  but  produce  revenue  for  the  state. 


Production  in  BCM  Overall  In  2025 


ALASKA  CANADA  MEXICO  US_land2  US_3and4  US_5and6  US_7  US_8and9 


■  Base 

■  MPEC 

■  MoreShale 

■  ShaleTax 

■  AIITax 


Figure  4.4:  Overall  Production  in  2025  as  Predicted  by  the  Model41 


41  US_land2,  for  example,  gives  data  for  US  census  regions  1  and  2  combined. 
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250000 


Producer  Profit  in  Millions  $  in  2025 


■  Base 

■  MPEC 

■  MoreShale 

■  ShaleTax 

■  AIITax 


N_US2  N_US3  N_US4  N_US5  N_US6  N_US7  N_US8 

Figure  4.5:  Producer  Profit  in  2025  as  Predicted  by  the  Model42 


Figure  4.6:  Shale  Producers  in  2025  as  Predicted  by  the  Model43 


42  N  US3,  for  example,  gives  profit  at  the  node  for  US  census  region  3. 

43  PU5S,  for  example,  gives  the  production  at  US  node  5  for  shale  gas. 
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Data  for  consumption  and  prices  (Figure  4.7  and  Table  4.11,  respectively), 
however,  show  that  the  producers  will  pass  most  of  the  tax  onto  the  consumers.  This 
also  shows  the  strength  of  the  World  Gas  Model,  by  predicting  which  areas  will  show 
a  change  in  prices.  Nodes  5,  and  6,  will  take  on  the  burden  of  the  tax  with  prices 
going  slightly  up  ($7.14/MMBTU  vs.  7.07  $/MMBTU)  and  consumption  relatively 
unchanged  when  compared  to  the  MPEG  case.  Nodes  1  and  2  contain  a  majority  of 
the  Marcellus  shale  play;  hence  prices  there  go  up  with  the  imposition  of  a  tax  on 
shale  gas.  Moreover,  US  nodes  7  and  8  have  high  production,  and  it" 's  profitable  for 
these  producers  to  sell  at  a  lower  price  in  their  own  market  and  at  a  higher  price  to  the 
other  nodes.  However,  imposing  a  tax  on  US  Node  7  increases  prices  at  that  particular 
node  in  2025  when  compared  to  the  MPEG  case.  Since  the  shale  producer  at  node  7  is 
the  Stackelberg  leader,  in  this  case  it  can  derive  more  profits  by  passing  the  tax  onto 
its  own  consumption  node.  Note  that  the  prices  under  the  two  tax  cases  at  node  7 
(4.94  $/MMBTU  and  5.13  $/MMBTU  in  the  ShaleTax  and  AllTax  case,  respectively) 
are  still  lower  than  the  price  for  the  Base  Case  (5.72  $/MMBTU,  when  the  shale 
producer  at  node  7  is  not  a  Stackelberg  player). 
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Consumption  in  BCM  Overall  In  2025 


ALASKA  CANADA  MEXICO  US_land2  US_3and4  US_5and6  US_7  US_8and9 


Figure  4.7:  Consumption  in  2025  as  Predicted  by  the  Model 
Table  4.11:  Average  Prices  in  S/MMBTU  in  2025 


Region 

Base 

MPEC 

MoreShale 

ShaleTax 

AllTax 

Alaska 

6.17 

6.17 

6.17 

6.17 

6.17 

Canada 

6.94 

6.69 

6.02 

6.50 

6.34 

Mexico 

6.66 

6.65 

5.63 

6.70 

6.92 

US  Nodes  1  &  2 

8.85 

8.91 

8.59 

8.98 

8.88 

US  Nodes  3  &  4 

7.48 

7.52 

7.02 

7.48 

7.37 

US  Nodes  5  &  6 

7.36 

7.07 

6.76 

7.14 

7.14 

US  Node  7 

5.72 

4.88 

4.59 

4.94 

5.13 

US  Nodes  8  &  9 

6.31 

6.03 

5.65 

6.01 

5.88 
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4.5.  Summary 


This  chapter  provides  a  novel  way  to  solve  mathematical  programs  with  equilibrium 
constraints.  The  new  method  has  been  shown  to  be  computationally  tractable,  and 
able  to  solve  MPECs  where  the  lower  level  is  a  complementarity  problem.  An 
extension  to  solve  EPECs  is  also  presented. 

The  method  was  first  applied  to  numerical  examples  for  MPECs.  It 
outperformed  the  method  of  disjunctive  constraints  in  two  ways.  First,  the  selection  of 
the  constant  L  for  Algorithm  4.1  did  not  prove  as  difficult  as  the  selection  of  the 
constant  K  in  disjunctive  constraints.  Second,  with  numerical  tests  the  method  proved 
to  be  computationally  quicker  than  the  method  of  disjunctive  constraints.  The  method 
was  also  shown  to  be  able  to  solve  a  numerical  example  of  an  EPEC,  but  extensive 
numerical  and  theoretical  results  will  be  part  of  future  work. 

The  method  was  applied  to  an  example  of  a  shale  gas  producer  in  the  US 
natural  gas  market  acting  as  a  dominant  player.  The  results  show  that  in  the  case  of  a 
Stackelberg  structure,  the  profits  of  the  producer  are  not  negatively  affected  with  the 
current  proposals  for  taxes.  However,  with  this  structure  the  producers  are  able  to 
pass  the  tax  onto  the  consumer,  as  profits  do  not  decrease  with  the  implementation  of 
tax  but  prices  do  go  up.  Moreover,  if  more  resources  are  present,  the  producer  is  able 
to  take  advantage  of  them.  While  in  actuality  the  Stackelberg  player  might  not  have 
such  an  advantage,  this  setup  helps  show  how  under  this  scenario,  producers  can 
manipulate  the  market  to  make  decent  decisions. 
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Chapter  5:  Solving  Discretely-Constrained  Mixed  Linear 
Complementarity  Problems 

5.1.  Introduction 

This  chapter  provides  solution  techniques  for  DC-MLCPs.  In  particular,  this  chapter 
will  consider  discretely-constrained  Nash  games  (DC-Nash),  where  some  of  the 
decision  variables  are  constrained  to  be  integer- valued.  These  games  have  been 
formulated  as  complementary  problems  (Cottle  et  ah,  2009)  in  the  literature. 
However,  the  discretely-constrained  versions  have  often  been  needed  to  be  solved 
using  inspection;  for  example,  a  bimatrix  game  table  (Gibbons,  1996)  which  has 
finite,  discrete  choices  to  choose  from. 

When  solving  DC-MLCPs,  it  is  important  to  realize  that  a  particular  instance 
might  not  have  integer  solutions.  A  compromise  would  be  to  get  solutions  that  are  as 
close  to  integer  as  possible.  While  this  chapter  does  not  provide  theoretical  arguments 
for  the  near-integer  solutions,  the  numerical  results  presented  corroborate  that  the  new 
technique  helps  achieve  integer  solutions  where  appropriate.  The  method  presented  in 
this  chapter  is  shown  to  be  better  than  the  method  of  (Gabriel  et  ah,  2011a),  (Gabriel 
et  ah,  2011b)  in  computational  effort  and  because  the  method  in  this  dissertation  does 
not  require  the  selection  of  a  constant  while  the  method  in  (Gabriel  et  ah,  2011a), 
(Gabriel  et  ah,  2011b)  requires  the  selection  of  a  specified  constant. 

First,  a  description  of  the  two  relaxation  conditions  will  be  given.  Then,  a 
general  formulation  to  solve  these  problems  will  be  provided,  based  on  the  work  of 
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(Gabriel  et  al.,  2011a),  (Gabriel  et  al.,  2011b).  Finally,  the  methods  from  Chapter  4 
will  be  applied  to  solve  this  resulting  two-level  problem  formulation  for  numerical 
examples  that  are  DC-Nash  games  and  discretely-constrained  network  problems. 

A  portion  of  this  chapter  has  been  presented  in  (Gabriel  et  al.,  2011a), 
(Gabriel  et  al.,  2011b).  However,  a  new  way  to  solve  these  problems  is  presented  in 
this  dissertation  which  was  not  used  in  the  aforementioned  papers.  Both  papers 
(Gabriel  et  al.,  2011a),  (Gabriel  et  al.,  2011b)  used  disjunctive  constraints  to  solve  the 
DC-MLCPs,  but  this  dissertation  uses  the  technique  of  SOS  Type  1  variables 
explained  in  Chapter  4.  The  examples  taken  from  (Gabriel  et  al.,  2011a),  (Gabriel  et 
al.,  2011b)  are  exactly  the  same  as  in  the  papers  but  the  solution  technique  is 
different.  An  extra  example  with  computational  time  is  provided  to  further  support 
the  use  of  Chapter  4  techniques  as  opposed  to  disjunctive  constraints  which  was  not 
discussed  in  the  two  papers  (Gabriel  et  al.,  2011a),  (Gabriel  et  al.,  2011b).  Hence,  all 
the  examples  in  this  chapter  were  solved  by  the  solution  technique  developed  in 
Chapter  4,  which  was  original  work  that  is  part  of  this  dissertation.  But  the  problem 
formulation  presented  in  this  chapter  and  the  theory  behind  the  formulation  was 
developed  in  two  papers  (Gabriel  et  al.,  2011a),  (Gabriel  et  al.,  2011b)  and  cannot  be 
regarded  as  original  work.  The  solution  technique  of  Chapter  4,  however,  proves  to  be 
computationally  superior  for  the  examples  presented. 
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5.2.  Discretely-Constrained  Mixed  Linear  Complementarity  Problems 


Recall  from  Chapter  2  that  a  general,  discretely-constrained 
complementarity  problem  is  given  the  vector  q  =  (<y, ,  £/2  )7 


A  = 


An  Al 


A21  A. 


,  find  z  =  (zj,  z2)r  e  S.R"  x  tR"2  such  that 


22  7 


mixed  linear 
and  matrix 


+(^n  A 

0  =  +  {Ai  A2I 


_L  z,  >  0 


27 
7  z  '\ 

zi 


1  z2 ,  z2  free 


VZlJ 

(zj)c  eiR+,ceC1,(z1)rf 

(z2  )c  e  9t+,  c  e  C2 ,  (z2  )rf  e  Z+ ,  d  e  D2 


(5.1) 


From  this  formulation,  if  (z,  )d  and  (z2  )d  were  continuous  variables,  the 
problem  would  simplify  to  a  linear  complementary  problem.  Since  they  are  not,  one 
obvious  solution  is  to  relax  them  to  be  continuous  variables,  and  then  solve  the 
problem  hoping  for  an  (approximate)  integer  solution.  However,  close  inspection 
shows  that  the  complementary  conditions  also  can  be  relaxed,  giving  another  option 
for  an  approximate  solution.  The  next  two  subsections  show  how  this  is  done  and 
follows  the  initial  problem  description  from  (Gabriel  et  ah,  2011a),  (Gabriel  et  ah, 
2011b). 


5.2.1  Epsilon-Integrality 

Consider  the  conditions  (z, )d  <e  Z,  ,c/  e  D,  and  (z2 )d  e  Z  ,d  eh,  from  (5.1).  Assume 
these  conditions  are  to  be  relaxed  to  make  this  problem  easier.  Without  loss  of 
generality,  consider (zx )d  e  Z+,d  e  Dx  as  the  arguments  for  (z2[eZ+,deh2  are 
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similar.  Then,  consider  a  small  deviation  shi,r  e  Dx,i  =  0,1,...,  A7  through  which  this 

discrete  variable  is  relaxed.  Given  any  feasible  set  M,  the  problem  becomes  to 
minimize  this  deviation  from  integrality  while  still  being  in  this  feasible  set.  This  is 
formulated  by  (5.2)  below 

N 

minX2>i,,l 

i=0  reD{ 

-  -V<  I  -"l„>  i  (z,  ),  -i  -  i  JV(1  -w„) 

i»w-l  (5.2) 

i= 0 

(ziieM 

r  e  Dvi  =  0,1,...,  V 

where  wlri  are  SOS  Typel  Variables 

In  (5.2),  the  integer  value  i  is  selected  that  is  closest  to  a  continuous  value  in 
M.  Note,  however,  that  the  objective  function  contains  a  nonlinear  function.  This 
absolute  value  function  can  be  decomposed  into  its  positive  part  and  negative  part  so 
that  the  objective  function  is  no  longer  nonlinear  as  in  (5.3). 

N 

minEE(f„)'  +(%,) 

i=0  re£>! 

-  N(  1  -wh,)  <  (z,  X  -  i  -  sWi  <  N(  1  -wM) 

£\ri  =  (£lri  )  _  (fl  ri  ) 

IX,  =1  (5-3) 

i=0 

(ziieM 

r  e  Dvi  =  0,1,...,  V 

where  wlri  are  SOS  Typel  Variables 

This  is  one  way  to  relax  the  DC-MLCP.  From  now  on,  this  will  be  referred  to 
as  {.'-integrality.  This  relaxation,  along  with  another  one  described  next,  will  be  used 
to  help  solve  DC-MLCPs. 
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5.2.2.  Sigma-Complementarity 


From  (5.1),  consider  the  complementary  condition 


0  —  <7i  +(/4 ii  -An ) 


.  Zj  >  0 


(5.4) 


"2  J 


This  condition  is  equivalent  to  (5.5)  below. 


<7i+(4i  At 

Zj  >  0 

(?i  +  (A  i  42) 


>0 


(5.5) 


rz  wT 

zi 


vz2yy 


z,  =  0 


The  goal  is  to  relax  the  last  line  equality  condition  in  (5.5).  To  obtain  a 
solution  that  is  approximate,  the  last  equality  need  not  equal  zero  but  can  be  very 
close  to  zero.  In  fact,  a  deviation  similar  to  the  one  for  integrality  can  be  developed 
here.  Consider  the  deviational  vector  'C,  such  that  the  relaxed  complementary  problem 
is  formulated  below44. 


+  C^ll  ^12 


>0 


-2  J 


Zj  >  0 


<h 


(5.6) 


\ 

fzA 

\ 

-a 

VZ2  J 

J 

(zj-<t)  =  0 


44  Note  that  different  f's  could  be  used  in  the  last  equation  in  (5.6)  as 


-C7,)=0' 


However,  this  relaxation  provides  a  same  solution  as  the  one  given  in  (5.6)  because  as  only  one  of  the 
factors  needs  to  be  0  for  the  product  to  be  zero. 
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Again,  the  problem  becomes  to  minimize  this  deviation  from  complementary 


whilst  still  being  in  this  feasible  set.  This  is  formulated  by  (5.7)  below 

rnin  \T <j 


<h 


+  {Aix  A12j 


>0 


Zj  >  0 


cr  >  0 


(5.7) 


\ 

fz A 

\ 

A,) 

-a 

\zi) 

J 

(zi  -  o')  =  0 


The  nonlinear  equality  condition  (a  product)  can  be  handled  the  same  way  the 
product  was  handled  in  Chapter  4.  Hence,  refonnulated  to  be  solved  with  SOS  Type  1 
variables,  (5.7)  becomes 

min  lrcr 


<h 


+  (^1,  A12) 


(  z  ^ 
zi 


>0 


VZ2  J 


Zj  >  0 

u  -  (v+  +  v_)  =  0 


<h 


+  (An  Ai)  1  -o  +(zi“^) 


u  = 


V^2  J  ) 


(5.8) 


(v+-vl  = 


7, 


r  „  \ 

\[  zi 

1 

-  cr 

VZ2  J 

) 

2 

-(zi-a) 


where  v+ ,  v  are  SO  S  1  variables 


5.2.3.  Complementarity,  Integrality  Trade-off 

One  of  the  advantages  that  relaxing  both  complementary  and  integrality  gives  is  that 
we  can  figure  out  the  tradeoff  of  relaxing  one  versus  the  other.  In  practice,  different 
solutions  can  be  achieved  depending  on  which  of  the  two  (or  both)  is  relaxed.  Figure 
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5.1  shows  the  idea  behind  this  tradeoff.  This  can  be  thought  of  as  the  Pareto  frontier 
of  a  multiobjective  programming  problem  (Cohon,  1978). 


s-integrality 


If  the  region  above 
the  tradeoff  curve 
contains  0  then  an 
exact  integer  and 
complementary 
solution  is  present. 


Tradeoff  Curve 
(Pareto  Frontier) 
for  Integrality  vs. 
Complementary 


C~ complementarity 

Figure  5.1:  The  Tradeoff  Between  Complementary  and  Integrality 


Although  the  whole  Pareto  frontier  might  not  have  a  smooth  convex  shape  as 
shown  in  the  figure,  it  is  motivation  enough  to  study  different  variations.  Moreover,  a 
point  on  the  tradeoff  does  not  necessarily  need  to  correspond  to  an  integer  solution  to 
the  DC-MLCP.  In  the  future  sections,  at  least  the  endpoints  of  the  tradeoff  curve  (the 
intersections  of  the  curve  with  the  axes)  will  be  calculated  to  give  an  indication  of  the 
extent  of  the  Pareto  frontier. 
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5.2.4.  Formulation  to  Solve  Discretely-Constrained  Mixed  Linear 
Complementary  problems 

The  following  fonnulation  is  from  (Gabriel  et  al.,  2011a)  for  solving  DC-MLCPs 
except  the  techniques  from  Chapter  4  have  been  used  instead  of  disjunctive 
constraints45. 


45  In  the  following  formulation  (5.8), 


{ww}i  are  SOS1  variables  implies  that  Wlrl  ,Wlr2  ,Wlr3 


are 


SOS1  variables. 


142 


mm 


co , 


IZ(£I„)'  +  (*lri)  +  £  Tj(£2ri)+  +{£2nY 

i= 0  reZ)j  /=— TVj  reZ)2 


+  (Or 


#1  +  (Ai  ^12 1 


>0 


2  7 


Zj  >  0 

u  —  (v+  +  v)  =  0 
^l  +  C^n  ^ 

„=v - 


-  lcr 


'27 


+  (zi-lar) 


(v+-v-)  = 


*7l  +  C^ll  ^12j 


-  lcr 


-27 


(zi-lo-r) 


0  q2  +  (2I21  ^22) 


-27 


where  v+ ,  v  are  SOS  1  variables 
-  N(  1  -Wln.)  <  (z, ),.  -  /  -  £lrf  <  tf(l  -ww) 

^Iri  =  (£\ri)  ~  (£\ri) 

(£lnYMriY^0 

N 

YjWM  =1 

z=0 


(iv) 


r  e  Dvi  =  0,1,...,  V 

where  {w^  J.  are  SOS  Ty pel  Variables 

-  JVj (1  -w2ri)  <  (z2 )r  -  i  -  s2ri  <  N2 (1  -w2ri) 

£lri  =  2ri  )  _  {£2 ri  ) 

(£2nYX£2riY  *  0 


i=-Nt 


r  €  D2,i  = 

where  {>v2f.f }.  are  SOS  Typel  Variables 


(5.8) 
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Here,  af1,af2  are  predetermined  weights  attached  to  ^-integrality  and  C- 
complementarity,  respectively.  These  weights  can  be  used  to  determine  the  tradeoff 
decision  between  ^-integrality  and  ^-complementarity. 

5.3.  Discretely-Constrained  Nash-Cournot  Games 

One  way  to  solve  Nash-Cournot  games  is  to  first  convert  them  to  complementary 
problems.  This  requires  that  the  objective  and  constraint  functions  be  differentiable 
and  that  the  KKT  conditions  can  be  fonnulated.  When  some  of  the  variables  are 
integer-valued  (e.g.,  binary  yes/no,  integer  production),  the  KKT  conditions  are  not 
valid  because  the  functions  are  no  longer  continuous.  This  section  shows  an  approach 
by  (Gabriel  et  ah,  2011b)  that  provides  a  compromise  between  complementarity  and 
integrality.  This  is  done  by  first  relaxing  the  discretely-constrained  variables  to  their 
continuous  analogs  and  taking  KKT  conditions  for  this  relaxed  problem.  Gabriel  et  al. 
(Gabriel  et  al.,  2011b)  converted  these  KKT  conditions  to  disjunctive-constraints 
form  (Fortuny-Amat  &  McCarl,  1981)  and  solved  them  along  with  the  original 
integer  restrictions  re-inserted  in  a  mixed-integer,  linear  program  (MILP).  The  integer 
conditions  were  then  further  relaxed,  but  targeted  using  penalty  terms  in  the  objective 
function.  This  MILP  by  (Gabriel  et  al.,  2011b)  relaxes  both  complementarity  and 
integrality  but  tries  to  find  minimum  deviations  for  both  and  as  such  is  an  example  of 
bi-objective  problem  (Cohon,  1978).  This  section  will  follow  the  same  methodology 
except  the  technique  of  SOS  type  1  variables  from  Chapter  4  will  be  used  instead  of 
disjunctive  constraints. 

The  advantage  of  the  technique  presented  in  this  chapter  over  the  fonnulation 
with  disjunctive  constraints  is  that  a  large  constant,  which  is  essential  for  the 
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formulation  outlined  in  (Gabriel  et  al.,  2011b),  does  not  need  to  be  selected.  Instead 
SOS  Type  1  variables  are  used  as  in  Chapter  4.  This  method  is  also  shown  to  be 
computationally  quicker  than  the  method  of  disjunctive  constraints,  and  numerical 
evidence  is  provided  later  in  this  chapter.  Note  that  for  all  numerical  tests  presented  in 
this  dissertation,  the  solutions  were  the  same  as  in  (Gabriel  et  al.,  2011a),  (Gabriel  et 
al.,  2011b). 

The  next  sections  provide  a  formulation  based  on  the  one  presented  in  section 
5.2.  Then,  numerical  results  for  two  different  discretely-constrained  Nash  games  are 
presented.  The  first  one  has  discrete  restrictions  on  the  production  quantities  while  the 
second  one  has  discrete  startup/shutdown  variables. 

5.3.1.  Formulation  of  a  DC-Nash  game  by  Gabriel  et  al.  (2011b) 

For  the  DC-Nash  game,  assume  there  are  several  Cournot  power  producers  that 
maximize  their  profit  simultaneously  by  choosing  their  optimal  production  quantities. 
Their  objective  function  (profit)  depends  on  the  production  of  the  competitors 
through  the  market  demand  curve  (relationship  between  the  total  production  and  the 
market  price)  as  well  as  their  own  marginal  cost.  Players  p  =  1  seek  optimal 
values  for  their  decision  vectors  xp  e  Xp,p  =  \,...,Pby  minimizing  their  cost 
functions  (or  negative  profit  functions)  fp(*,x"p)  such  that 
fp(xp,xp)<  fp(xp,x~p),Vxp  eXp  (5.9) 

Here  xp  e  9?”  represents  the  variables  under  player  p"s  control  with  x~p  the 
remaining  variables  for  other  players.  Also,  xp  means  an  equilibrium  value  to  xJ>  and 
Xp  =CP  n  Zn+P  where 
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(5.10) 


Cp  =  |  g pj  (xp  )  <  0,  j  =  1, . . I p ;  hp  (xp  )  =  0,  k  =  1,. . . ,  E 


* 

p’^q 


>0,qeSp} 


and  Z"p  is  the  set  of  nonnegative,  integer-valued  variables  ,  i.e., 

xp  eZ+)re{l,..,n  }\5  .  Here  Sp  represents  those  indices  for  xJ>  that  relate  to 
continuous  variables.  A  continuous  relaxation  would  then  be  to  replace  X?  by  Cp,  i.e., 
find  xp ,  p  =  1 , . . . ,  P  such  that 

fp(xp,x~p)<  fp(xp,x~p),Vxp  eCp  (5.11) 

or  equivalently  find  xp  that  solves 
min  fp(xp,xp) 

xp 

S.t. 

gp(xp)<0 

K(xp)  =  0  (5.12) 

j  —  f  •••>  Ip  ■>  k  —  !?•••?  E  p 
xp>0 
q  e  Sp 

For  the  Karush-Kuhn-Tucker  (KKT)  conditions  of  (5.12)  to  be  equivalent  to  solving 
that  optimization  problem,  the  assumption  that  the  functions  fp(»,x~p )  are  convex 
and  a  constraint  qualification  (see  (Bazaraa  et  ah,  1993)  for  generalization  of  these 
assumptions  that  will  also  lead  to  KKT  conditions  being  sufficient  for  optimality) 
holds  (e.g.  gp(xp),hp (xp)  linear)  is  needed.  The  KKT  conditions  for  player  p's 

relaxed  problem  (5.12)  are  to  find  xp  e  9{"P,AP  e  9? , yp  e  9? E,‘  such  that 
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0  <  V x\x~f)  +  Xvgf(x')^  +  ZvV(x')r;  -L  x’  >  0 

j<Elp  keEp 

0<-gp(xp)±Ap>0 

j  =  h-,Ip  (5-13) 

0  =  hp(xp),yp  free 

k  =  l,...,Ep 

Gabriel  et  al.  (2011b)  showed  that  the  solution  to  (5.13)  with  the  discrete  restrictions 
inserted  back  is  the  same  as  the  solution  to  (5.9). 

To  be  able  to  end  up  with  a  linear,  mixed-integer  program  the  payoff  function 


fP(xp,x-p) 


1 

(  P  \ 
xp 

T 

' Np  Np ' 

(  P  \ 
xp 

2 

xp 

J 

{Np  Np , 

x  p 

J 

+  (cp)Txp 


is  restricted  to  be  quadratic  and 


the  constraint  functions  to  be  linear.  The  KKT  conditions  are 


o  <  v  Y Vg'u'M'  +  Y  vA/CiOn'  J- a  0 

jslp  k^Ep 

gp(xP)  =  (dPJxP  ~KP  <  0 

7=1,...,/,  (5.14) 

/^(xo=k)r^-^=o 

k  =  \,...,Ep 

Gabriel  et  al.  (2011b)  refonnulate  the  continuous  relaxation  of  the  original 
problem  (5.9)  by  using  the  complementarity  problem  form  of  the  Nash  problem 
suitably  relaxed  as  in  (5.13).  These  KKT  conditions  are  equivalent  to  a  set  of 
disjunctive  constraints  of  the  form: 
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0 <  vi,/'(x',x-')+  £vg fix')*;  +  Y,vV(xr)r£  s  JW 

;e/p  (te£p 

o 

o<  A/;(x/j)  =  ^(i-</) 

7  =  1,...,/,  (5.15) 

0  =  h^{xp),ypk  free 
k  =  l,...,Ep 

u{  e  {0,l}'!p 
wf  e  {0,l}/p 

for  suitably  large  values  of  and  K?  that  can  be  computed  as  described  in  (Gabriel 
et  al„  201  lb). 

An  alternative  method  is  to  use  SOS  Type  1  variables  as  described  in  Chapter 
4,  which  will  be  used  here  so  the  suitably  large  values  of  Kf  and  Aff  do  not  need  to 
be  computed.  This  is  described  below 
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0  <Vx,f’(x',x-’)+  Xvg'(xr)A'  + 

jelp  k<EEp 

0<  xp 


vx,ru'’,x-’)+  +  £  VA'Oc'K  +  *' 


u!’  =  ■ 


Mp 


keEp 


V, +  Ivg'(.r'u;  +  XV*'(^W 
/  p  V  /  p  \  7‘e/^  ^eis^ 

I'l  l  -KJ  = - 2 - 

<-(<)*-(<)"=» 

where  (vf )+  -  (v,/J )  are  SO  S  1  variables 

o<-g;(xO 

0  <  Ay  (X^) 

,  _-g;(xo+^(xo 


«V  _ 


kJ-fe)‘=o 


0  =  hp(xp),y[  free 
k  =  l,...,Ep 


(5.16) 


Using  the  quadratic  form  o  l'//)  and  the  linear  forms  of  gp  and  h  p  from  above, 
results  in  the  following  linear,  mixed-integer  (with  SOS1  variables,  that  are  defined 

p 

using  integers)  program  with  arbitrary  objective  function  ^(zp)7 xp  and  the  integer 

p= i 


restrictions  added  back: 
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^  />=! 

si.  for  all  p  =  1 

N'  +  N^'  +  UNf  +  N'^+c'  +  Y.dp'j  +  lefrf 

Z  Z  jelp  k^Ep 

0<xp 


'-{n;  +  N?y  +  '-{n;  +  N[T\-’ +c'  +  yrf'A'  +  Y.e’r’t 


Mp 


k&Ep 


i  (»,"  +  W,''  ^  +  i  (tf'  +  Nf  \-p  +  <•'  +  £  rfp'  +  X  *'  -  ■ 

^  ^  j^Ip  kcEp 


<-(vf)+-(vf)"=  o 

where  (vf )+ ,  (vf )  are  SOS  1  variables 

OS-t^jV-v) 

0<^(xp) 


M2J  =  ■ 


((^V-^j+^xQ 


[vrJ  _kj)- 


-fe)'=o 

where  (vf . )+ ,  (v£ . )  are  SO  S 1  variables 
i  =  Wp 

0  =  (e*  fx'1  -8P  ,yl  free 


(5.17) 


Note  that  the  above  problem  requires  integral  restrictions  and  complementary 
restrictions  to  hold  at  the  same  time,  and  may  prove  to  be  infeasible  (Gabriel  et  al., 
2011b).  This  is  the  crucial  conversion  to  a  two-level  problem.  In  (5.17),  the  upper- 
level  has  an  objective  function  that  is  arbitrary.  Hence,  (5.17)  is  essentially  still  a  one- 
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level  problem,  with  only  the  Nash-Cournot  game  at  the  bottom  level  refonnulated 
with  SOS  1  constraints  being  equivalent  to  solving  a  complementary  problem. 

The  one-level  complementary  problem  can  be  infeasible,  so  it  needs  to  be 
relaxed.  The  relaxations  introduced  are  the  Epsilon-Integrality  (Section  5.2.1)  and 
Sigma-Complementary  (Section  5.2.2)  for  the  problem  to  be  feasible.  Minimizing 
these  deviations  can  be  put  in  the  objective  function,  making  this  one-level  problem  a 
two-level  problem.  The  lower-level  solves  a  relaxed  DC-Nash  game  while  the  upper- 
level  minimizes  the  deviations  from  complementary  and  integrality.  To  ensure  that 
the  above  refonnulation  does  not  have  a  conflict  between  complementarity  and 
integrality  46,  the  following  relaxed  version  of  the  problem  is  employed. 


46  We  assume  the  relaxed  continuous  version  of  the  problem  is  feasible. 
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it  xk.rt*k.rt  U2(ir(<+<)) 

V.  P=1  i=0  } 
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0  zHnp  +  NpT\p  +%{  +  Nf\~p  +  cp  +  ZdfVj  +  Yjelrl 
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Mip 
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keEp 
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where  (vf )+ ,  (vf )  are  SO  S 1  variables 

o^-kr 

0<A/}  (xp) 


j*ir 


kzEp 


<  = 


2 

(v'  )* - (vf )- , -k),-«',-«-,,)-<-kW)-^) 

where  (vf_ ,• )+ , (v|'; )  are  SO S 1  variables 
j  ~  !?•••?  Ip 

0  =  (e^  J  x7’  -8P ,yp  free 
k  =  l,...,Ep 

xr  eZ+,re\^...,np}\Sp 
cr/kcrf  >0 


(5.18) 


where  the  restrictions  on  the  relaxation  variables  are  the  same  as  in  (5.8)  and  given  by 
(5.19) 
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(5.19) 


-  N(1  -wlri)  <  {xp  l  -  i  -  (ejf  <  N(  1  -wM) 
i^riY  ={{^n)P)+  ~{{^n)P) 

N 

Zwi- =  l 
1=0 

r  e  =  0,1,...,  TV 

where  {wlr(.};  are  SO  SI  Variables 

In  the  above  formulation  (5. 1 8)-(5. 19),  the  (slriY  are  used  to  target  the 
specified  integer  values  (e-integrality)  and  a/,  of  are  used  to  relax  complementarity 
(^-complementarity),  both  of  which  are  minimized  in  the  objective  function 
weighting  the  two  objective  function  parts  with  positive  weights  and  co2.  Thus, 
minimizing  these  deviations  helps  find  an  optimal  integer  solution,  as  described  in 
(Gabriel  et  ah,  2011a). 

5.3.2.  First  Numerical  Example 

This  section  presents  the  results  of  numerical  examples  for  solving  discretely- 
constrained  Nash-Cournot  games  from  the  theory  outlined  in  the  previous  subsection. 
The  first  example  constrains  the  production  quantities  to  be  integer  while  the  second 
example  has  continuous  production  quantities  but  binary  startup/shutdown  variables. 
In  both  examples,  seven  variations  are  considered.  These  variations  go  through 
different  relaxation  techniques  and  combinations  of  fonnulations  to  be  described 
later.  The  problems  selected  can  be  shown  to  have  unique  solutions  by  simple 
algebra. 

The  results  show  that  fonnulation  (5.18)-(5.19)  provides  solutions  to  the 
original  discretely-constrained  problems.  The  variations  also  show  that,  as  stated 
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before,  (5.17)  can  lead  to  an  infeasible  solution.  Moreover,  relaxing  complementarity 
in  (5. 1 8)-(5. 19)  but  keeping  integer  restrictions  also  leads  to  a  discrete  feasible 
solution.  Both  numerical  examples  show  that  relaxing  complementarity  is  essential  to 
obtaining  discrete  solutions.  Enforcing  discrete  restrictions,  even  by  integer 
relaxation,  does  not  help  obtain  the  integer  solutions  and  relaxation  of  complementary 
conditions  is  necessary.  A  combination  of  both,  as  presented  in  (5 . 1 8)-(5 . 1 9)  helps 
obtain  the  required  solutions  in  both  cases. 

For  ease  of  presentation  and  comparison  but  with  no  loss  of  generality, 
consider  a  Nash-Cournot  game  with  two  players  (p  =  1,2).  Given  an  inverse  demand 
curve  Price  =  a  -  b(Quantity),  each  player  chooses  qp  e  Z+  to  maximize  their  profit 
function 

Profit  p  =  Price  xqp-  (fipq]  +  ppqp  )  (5.20) 

where  the  term  in  parentheses  denotes  cost  as  a  function  of  quantity  selected  i.e.,  qp. 
The  formulation  of  the  game  is  the  same  as  discussed  in  the  previous  subsection. 

For  the  first  example,  let  a  =  6,  b  =  1 ,  =  /?2= 1,  and  pi=p2=l,  as  well  as  adding 

capacity  constraints  for  both  players  of  the  fonn 

< iP^q >*  (5-21) 

where  qmax  =  4.  Since  only  integer-valued  production  qp  is  allowed,  a  bimatrix  payoff 
table  (assuming  maximizing  payoff)  as  shown  below  in  Table  5.1  is  employed  to 
solve  (5.9). 
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Table  5.1:  Bimatrix  Nash-Cournot  Game,  Profits (qdqi) 


0 

1 

2 

3 

4 

0 

(0,0) 

(0,3) 

(0,2) 

(0,-3) 

(0,-12) 

1 

(3,0) 

(2,2) 

(1,0) 

(0,-6) 

(-1,-16) 

2 

(2,0) 

(0,1) 

(-2,-2) 

(-4,-9) 

(-6,-20) 

3 

(-3,0) 

(-6,0) 

(-9,-4) 

(-12,-12) 

(-15,-24) 

4 

(-12,0) 

(-16,-1) 

(-20,-6) 

(-24,-15) 

(-28,-28) 

Clearly  qj  =  q2  -  1  is  the  unique  Nash  equilibrium  in  pure  strategies.  Another 
way  to  solve  Nash-Cournot  games  is  by  simultaneously  solving  the  problems 

max  [a  -  b(g1  +  q2)]qp  -  (fipq2p  +  ppqp ) 

qp 

S'L  (5.22) 

Vp  ^  4W  (Ap  dual) 

qP>  0 

for  p  =  1,2.  Since  the  slope  of  the  inverse  demand  function  b  >  0  and  (:lp  >  0,  the 
KKT  conditions  are  both  necessary  and  sufficient  for  solving  these  problems.  These 
conditions  are  to  find  qi,q2MM  that  solve  the  following  linear  complementary 
problem  (LCP): 

0<2  qp(b  +  pp)  +  bq_p-(a-pp)  +  Ap  Lqp>  0 
§<qimL-qpLXp>  0 

for  each  p  =  1,2.  However,  the  KKT  conditions  are  only  valid  if  qp,  p  =  1,2  are 
continuous-valued.  Thus,  the  resulting  LCP  needs  to  avoid  discrete  restrictions  on  the 
qv  variables.  In  this  particular  example,  solving  the  above  LCP  after  assuming 
qp  e  V.H .  results  in  the  integer  solution  c/i  =  1 ,  q2=  1  with  Price  =  4. 
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However,  changing  some  of  the  data  to  a  =  9  and  p2  =  3  results  in  a  non¬ 
integer  solution  of  q\  =  1.733,  q2  =  1.067,  and  Price  =  6.2.  But  the  new  bimatrix 
payoff  table  for  the  original  discrete  version  of  this  game  with  these  new  data  (Table 
5.2),  shown  below,  gives  a  unique  discrete  solution  of  =  2,  q2  =  1  with  Price  =  6. 


Table  5.2:  Nash-Cournot  Game,  Profits (qilqi),  (Only  Adjustments  a=9,  p2  =  3) 


0 

1 

2 

3 

4 

0 

(0,0) 

(0,4) 

(0,4) 

(0,0) 

(0,-8) 

1 

(6,0) 

(5,3) 

(4,2) 

(3,-3) 

(2,-12) 

2 

(8,0) 

(6,2) 

(4,0) 

(2,-6) 

(0,-16) 

3 

(6,0) 

(3,1) 

(0,-2) 

(-3,-9) 

(-6,-20) 

4 

(0,0) 

(-4,0) 

(-8,-4) 

(-12,-12) 

(-16,-24) 

This  example  shows  what  can  happen  if  the  relaxed  LCP  does  not  provide 
integer-valued  answers.  Next,  more  numerical  tests  are  described  with  the  new  data  a 
=  9  ,b  =  1,Ai=A2  =  1,Pi=  l,andp2  =  3. 

The  first  variation  is  to  solve  the  continuous  version  of  the  LCP  (i.e.,  without 
any  integer  restrictions)  relating  to  (5.9)  ("MLCP").  Solving  the  original  version  of 
the  problem  with  the  integer  restrictions  relating  to  (5.9)  is  variation  2  ("Bimatrix") 
and  is  solved  by  examining  the  bimatrix  payoff  table.  In  the  remaining  variations  to 
be  described,  there  are  two  ways  of  forcing  integrality  of  the  solutions.  First,  the 
problem  can  be  integer-constrained  through  the  solver  (variations  3  and  4)  with 
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variation  3  being  (5.17)  and  variation  4  also  relaxing  complementarity  (C- 
complementary)  in  (5.17). 

Second,  in  variation  5,  complementarity  can  be  relaxed  without  constraining 
the  problem  to  have  integer  solutions,  hence  "continuous  variables"  for  the  problem 
description.  Hence,  we  should  not  expect  integer  solutions.  Finally,  in  variations  6 
and  7,  integers  can  be  targeted  using  the  e  deviational  variables  (5. 1 8)-(5. 1 9)  (e- 
integrality).  In  variation  6,  no  relaxation  for  complementarity  is  allowed.  Variation 
7  allows  relaxation  for  both  complementarity  and  integrality  (^-complementary  and  e- 
integrality).  Table  5.3  describes  the  various  possible  fonnulations  considered. 


Table  5.3:  Description  of  Formulation  Variations 


Variation 

C-C  omplementary 

^-Integrality 

Problem  Description 

1 

No 

No 

MLCP 

2 

No 

No 

Bimatrix 

3 

No 

No 

Integer  variables 

4 

Yes 

No 

Integer  variables 

5 

Yes 

No 

Continuous  variables 

6 

No 

Yes 

Continuous  variables 

7 

Yes 

Yes 

Continuous  variables 

5.3.3.  Results  for  First  Numerical  Example 

Tables  5.4  and  5.5  below  give  the  results  for  this  first  numerical  example. 
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Table  5.4:  Summary  of  Results  (a  =  9,  b  =  1,  Pi=  p2  =  l,/>i  =1,  p2  =  3) 


Variation 

Solution  (qi,q2) 

Price 

Profits  (Pi,  P2) 

1 

(1.733,1.067) 

6.2 

(6.01,2.28) 

2 

(2,1) 

6 

(6,2) 

3 

Infeasible 

Infeasible 

Infeasible 

4 

(2,1) 

6 

(6,2) 

5 

(1.733,1.067) 

6.2 

(6.01,2.28) 

6 

(1.733,1.067) 

6.2 

(6.01,2.28) 

7 

(2,1) 

6 

(6,2) 

Table  5.5:  Summary  of  Results  (a  =  9,  b  =  1,  Pi=  p2  =  l,/>i  =1,  p2  =  3) 


Variation 

Sum  £ 

Sum 

1 

n/a 

n/a 

2 

n/a 

n/a 

3 

n/a 

n/a 

4 

n/a 

0.2 

5 

n/a 

0 

6 

0.334 

n/a 

7 

0 

0.2 

Table  5.4  shows  that  a  solution  to  the  integer-constrained  Nash  game  is  to 
have  cj-l  =  2,  q2  =  1  with  a  resulting  price  of  6  (variation  2).  When  the  integer 
restrictions  are  removed,  the  solution  is  then  q1  =  1.733,  q2  =  1.067  with  the  new 
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price  of  6.2  (variation  1).  Solving  the  mixed  integer  programming  (MIP)  version  of 
the  problem  but  forcing  exact  complementarity  and  integrality  results  in  an  infeasible 
solution  (variation  3)  as  would  be  expected.  Interestingly,  the  original  integer 
solution  to  the  Nash  problem  can  be  obtained  with  the  MIP  approach  as  long  as 
complementarity  is  relaxed  (variation  4)  or  when  integers  are  targeted  using  e’s 
(without  enforcing  integrality)  along  with  the  complementarity  relaxation  (variation 

7). 

It  is  interesting  to  note  that  variation  7  is  a  validation  of  the  earlier  discussion 
for  obtaining  integer  solutions  to  DC-Nash.  From  the  perspective  of  accuracy  in 
attaining  the  original  production  values  and  price,  the  MIP  approach  is  correct  in  this 
instance  and  thus  provides  an  alternative,  viable  method  for  solving  such  problems.  It 
is  interesting  to  note  the  difference  in  results  between  variations  4  and  5.  The  former 
achieves  the  correct  integer  solution  but  directly  forces  the  variables  in  GAMS  to  be 
integer-valued.  The  latter  allows  relaxation  of  complementarity  but  does  not  give 
integer  solutions  as  expected.  Furthermore,  variation  6  also  does  not  get  the  correct 
integer  solution  even  though  the  using  the  s  deviational  variables  were  included. 

To  compare  computational  time,  the  fonnulation  of  the  numerical  example 
above  was  expanded  where  the  number  of  players  P  was  increased  but  the  marginal 
cost  for  half  the  players  was  set  the  same  as  player  1  and  the  other  half  the  same  as 
player  2  from  the  above  example  for  variation  7.  The  following  Figure  5.2  shows  this 
result  with  an  increase  in  the  number  of  players  for  the  method  of  this  dissertation 
compared  to  the  method  by  (Gabriel  et  al.,  2011b).  Again,  the  method  of  disjunctive 
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constraints  is  computationally  slower  for  this  example  when  compared  to  the  SOS  1 
method. 


ComputationalTime  to  Solve  for  Number  of  Players  (First 
Numerical  Example  DC-Nash  Game) 


- SOS  1  (Method  of  Chapter  4) 

- DisjCons  (Gabriel  et  al.,  2011b) 


Figure  5.2:  Computational  Time  for  First  Numerical  Example 


5.3.4.  Numerical  Example  Relevant  to  Production  Systems 

In  many  applications,  the  quantities  qp  are  actually  positive  real  numbers  but  there  are 
also  constraints  of  the  form 

s  q  <q  <  s  q  (5.24) 

where  sp  is  a  binary  variable  that  is  1  when  the  player  p  chooses  to  produce  and  0 
when  player  p  chooses  to  not  produce.  Here  the  binary  variable  sp  might  for  example 
relate  to  the  on/off  status  for  a  power  generation  unit.  If  on,  then  the  minimum  and 
maximum  production  quantities  are  in  force.  If  off,  then  both  the  upper  and  lower 
bounds  are  equal  to  zero. 

The  original  capacity  constraint  is  replaced  by  the  one  above  and  the  resulting 
Nash-Cournot  game  is  then  solved  with  a  =  9,  b  =  1,  =  f}2  =  1,  p\  =  1,  p2  =  3,  qmin  = 
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1.5,  and  qmax  =  4.  The  binary  variables  sp  are  the  ones  targeted  when  complementarity 
and  integrality  are  relaxed  but  still  allowing  for  continuous  generation  variables.  The 
following  tables  summarize  the  results. 


Table  5.6:  Summary  of  Results  (Example  Relevant  to  Production  Systems) 


Variation 

Solution  (q\,q2) 

Binary(s/,  s2 ) 

Profits  {PI,  P2 ) 

1 

(1.733,1.067) 

(0.347,0.213) 

(6.01,2.28) 

2 

(1.625,1.5) 

(U) 

(5.28,  2.06) 

3 

(1.625,1.5) 

(U) 

(5.28,  2.06) 

4 

(1.625,1.5) 

(U) 

(5.28,  2.06) 

5 

(1.733,1.067) 

(0.347,0.711) 

(6.01,2.28) 

6 

(1.625,1.5) 

(U) 

(5.28,  2.06) 

7 

(1.625,1.5) 

(U) 

(5.28,  2.06) 
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Table  5.7:  Summary  of  Results  (Example  Relevant  to  Production  Systems) 


Variation 

Price 

Sum  e 

Sum  C 

1 

6.2 

n/a 

n/a 

2 

5.875 

n/a 

n/a 

3 

5.875 

n/a 

n/a 

4 

5.875 

n/a 

0 

5 

6.2 

n/a 

0 

6 

5.875 

0 

n/a 

7 

5.875 

0 

0 

The  solutions  to  this  example  are  very  different  from  the  previous  one. 
Variation  2  shows  the  true  solution  when  the  variables  sp,p=  1,2  are  forced  to  be 
binary.  Namely,  player  2  produces  at  the  minimum  level  of  1.5  but  player  1  chooses 
a  value  of  1.625,  in  between  the  minimum  and  maximum.  The  continuous  relaxation 
(variation  1)  achieves  higher  profits  for  both  players  as  would  be  expected  due  to  less 
restrictive  constraints  but  does  not  end  up  with  binary  values  for  the  sp  variables. 
Interestingly,  all  other  variations  on  relaxation  are  able  to  achieve  the  correct 
production  quantities  (qp)  and  binary  production  indicators  (sp)  except  for  variation  5 
when  only  complementarity  is  relaxed.  For  this  particular  problem,  forcing 
integrality  is  key  (through  one  of  the  two  aforementioned  methods)  as  variations  3,  4, 
6,  and  7  all  give  the  correct  binary  solution  for  sp,p=  1,2. 

Similar  to  the  previous  example,  the  number  of  players  was  increased  so  that 
half  the  players  had  the  data  for  player  1  and  the  other  half  for  player  2.  The 
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following  Figure  5.3  shows  this  result  with  an  increase  in  the  number  of  players  for 
the  method  of  this  dissertation  compared  to  the  method  by  (Gabriel  et  al.,  2011b), 
both  for  variation  7.  Again,  the  method  of  disjunctive  constraints  is  computationally 
slower  for  this  example  when  compared  to  the  SOS1  method.  However,  this  time  the 
advantage  of  SOS1  is  not  as  strong  as  for  the  first  numerical  example  in  the  previous 
section.  A  reason  for  this  can  be  that  since  the  decision  variables  are  binary, 
formulating  as  SOS  1  might  not  have  that  much  of  an  advantage.  This  contrasts  with 
the  first  example  where  the  decision  variables  were  integer. 


ComputationalTime  to  Solve  for  Number  of  Players  (Example 
Relevantto  Production  Systems) 


- SOS  1  (Method  of  Chapter  4) 

- DisjCons  (Gabriel  etal.,  2011b) 


Number  of  Players 


Figure  5.3:  Computational  Time  for  Example  Relevant  to  Production  Systems 


5.4.  Discretely-Constrained  Network  Problems 

This  section  considers  discretely-constrained  network  problems.  Note  that  these 
problems  can  be  cast  as  DC-Nash  games  as  well  (Cottle  et  al.,  2009).  However,  it  is 
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instructive  to  look  at  these  network  examples  separately  as  well,  as  extra  intuition  can 
be  gained  from  considering  transmission  lines.  The  first  example  is  a  continuation 
form  the  previous  section,  with  two  producers.  The  second  example  has  four 
producers  over  two  nodes. 

5.4.1.  First  Network  Example 

Consider  a  power  market  with  two  producers  supplying  to  one  demand  node  as 
shown  in  Figure  5.4.  Producers  1  and  2  choose  to  produce  quantities  tj\  and  q2 
respectively,  and  supply  it  to  meet  inelastic  demand  d,  while  there  are  transmission 
lines  (with  flow  variables  q12,  q±s,  <723)  between  the  three  nodes.  There  is  a  marginal 
utility  of  demand  q  and  marginal  costs  Ci  and  c2  for  producers  1  and  2,  respectively. 
There  is  also  a  market  operator  who  maximizes  its  own  profits  by  buying  from  the 
producers  and  selling  to  the  consumers. 
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Figure  5.4:  Diagram  of  First  Network  Example 


The  producer  p(p=  1,2)  solves  the  following  optimization  problem 


min) cpq 


P 


s.t. 

0  -  -  #max  (dual  fir  ) 


where  X„  is  the  (endogenous)  price  at  node  n.  Note  that  the  producer  p  is  active  at 
node  n  =p. 

The  market  operator  solves  the  following  optimization  problem  (with 
qp  introduced  to  have  a  square  system).  The  equality  constraints  set  the  power  flow 

(q13  for  example,  signifies  flow  from  node  1  to  node  3)  equal  to  the  power  produced 
and  the  inequality  constraints  give  a  bound  on  the  maximum  amount  of  flow  allowed. 
Flow  can  be  towards  the  opposite  direction  as  well  which  is  signified  by  a  negative 
number  (i.e.,  if  q13  is  negative,  then  the  flow  is  from  node  3  to  node  1),  so  the 
inequalities  contain  a  maximum  negative  flow  as  well. 

min  {c'| q ,  +c2q2  ~cdd\ 

<1\  >4 2^  >#13  >#2 3  2 

s.t. 

^1  =0  (A) 

^23  *?12  _  *7 2  ^  (A) 

d  —  qn  +  q2T,=Q  (A)  (r  ~>r\ 

(5.26) 

min  /  ^  max  /  /omin  nniax  \ 

~<ll2  ~  Q\2  ~  Q\1  ( Pl2  ?  P\  2  ) 

-q™<qn<q™  (At".  AD 

-^23  ^23^23  (/A  ,/A  ) 

<7l  =  ?1 

A  =q2 

The  above  optimization  problems  can  be  combined  to  form  an  MCP,  which 
gives  a  solution  to  the  game.  Our  goal  here  is  to  see  if  we  restricted  the  quantities 
produced  and  flows  to  be  integer-valued,  if  we  can  come  up  with  an  equilibrium 
solution.  The  following  Table  5.8  gives  the  values  of  the  parameters  used  for  solving 
this  network  problem. 
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Table  5.8:  Parameter  Values  Used  in  First  Network  Example 


max 

<h 

max 

<h 

max 

<hi 

9iT 

42T 

Cl 

C2 

Cd 

18 

20.5 

12 

15 

15 

2 

1 

5 

Hence,  producer  2  has  a  lower  marginal  cost  so  will  attempt  to  supply  more 
units  of  q2.  We  use  the  same  process  as  in  the  previous  section  and  fonnulate  the 
problem  according  the  variations  in  Table  5.9.  Note  that  we  are  not  considering  the 
bimatrix  game  for  this  example,  so  there  is  no  variation  2.  Table  5.10  shows  the 
results  for  the  example  under  different  variations. 


Table  5.9:  Description  of  Formulation  Variations 


Variation 

^-Complementary 

e-Integrality 

Problem  Description 

1 

No 

No 

MLCP 

3 

No 

No 

Integer  variables 

4 

Yes 

No 

Integer  variables 

5 

Yes 

No 

Continuous  variables 

6 

No 

Yes 

Continuous  variables 

7 

Yes 

Yes 

Continuous  variables 
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Table  5.10:  Solution  to  Power  Market  Example 


Variations 

1 

3 

4 

5 

6 

7 

qi 

9.5 

Infeasible 

10 

9.5 

9.5 

10 

92 

20.5 

Infeasible 

20 

20.5 

20.5 

20 

</l2 

-5.5 

Infeasible 

-5 

-5.5 

-5.5 

-5 

cll3 

15 

Infeasible 

15 

15 

15 

15 

<?23 

15 

Infeasible 

15 

15 

15 

15 

X\ 

2 

Infeasible 

2 

2 

2 

2 

X2 

2 

Infeasible 

2 

2 

2 

2 

X3 

5 

Infeasible 

5 

5 

5 

5 

d 

30 

Infeasible 

30 

30 

30 

30 

Sume 

n/a 

n/a 

n/a 

n/a 

1 

0 

Sum  C 

n/a 

n/a 

0.5 

0 

n/a 

0.5 

Note  that  again,  variation  7  gives  an  integer  solution.  Comparison  to  variation 
4  is  critical,  as  both  of  them  give  the  same  solution.  However,  variation  7  provides 
integer  solutions  but  does  not  explicitly  enforce  integrality,  while  variation  4  requires 
imposing  integer  restrictions  to  get  to  the  answer.  Variation  3  proves  to  be  infeasible, 
while  variations  5  and  6  show  that  only  including  a-complementarity  or  only 
including  s-integrality  is  not  sufficient  to  achieve  an  integer  solution  for  all  the 
variables  that  are  constrained  as  such.  Note  that  prices  at  each  node  (/li,  X2,  23)  stay 
the  same  at  each  node,  regardless  of  the  variation.  However,  variation  3  did  not 
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provide  any  solution,  so  not  only  does  variation  7  provide  an  integer  solution;  it  does 
so  without  imposing  integer  restrictions  and  also  delivering  reasonable  prices. 


5.4.2.  Second  Network  Example 

The  next  example  is  from  (Gabriel  et  ah,  2011a)  and  depicts  an  equilibrium  in  an 
energy  network  (e.g.,  natural  gas,  electricity)  where  production,  consumption,  and 
transmission  of  the  energy  product  are  analyzed. 

Four  energy  price-taking  producers  (A,  B,  C,  D)  are  modeled  with  the  first  two 
located  at  node  1  and  the  latter  two  at  node  2.  The  production  levels  are  denoted  as 
qpn  where  node  n  £{1,2}  and  producer  p  G  {A,  B,  C,  D).  Similarly,  the  sales  levels 


are  denoted  as  sp .  Lastly,  at  node  1 ,  the  two  producers  A  and  B  have  the  additional 


option  of  sending  energy  to  node  2  and  fxi ,  ff2  represents  the  associated  amounts  of 
flow.  (Note  that  the  producers  at  node  2  are  not  allowed  to  ship  their  product  to  node 
1.) 


Both  producers  A  and  B  at  node  1  have  structurally  a  similar  optimization 
problem  shown  below  just  for  producer  A.  For  node  2,  the  producers  have  an 
optimization  that  is  almost  the  same  as  at  node  1  with  the  exception  that  no  flow 
variables  (nor  related  terms)  are  included. 


max 


!^\s\  +nif\2  ~ci{cl'ii)~{Tr2G  +  T 12) fu\ 


s.t. 

qf<qf  (Af) 

s?=q?-At  (Sf) 


sf>  0 


qf  >  0 
fn  *  0 


(5.27) 
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where 


•  Kn  is  the  producer  price  at  node  nE  { 1,2} 

•  c^(^) is  the  (marginal)  production  cost  function  assumed  to  be  linear,  i.e., 


•  z^G  represents  the  nonnegative,  regulated  tariff  for  using  the  network  from 
node  1  to  node  2;  r/fG  is  a  fixed  parameter. 

•  //12  is  the  congestion  tariff  for  using  the  network  from  node  1  to  node  2  and  a 
variable  from  another  part  of  the  equilibrium  model 

•  qf  is  the  maximum  production  quantity47 

Each  producer  is  maximizing  their  profit  (5.27)  by  choosing  appropriate 
nonnegative  levels  of  production,  sales  and  flow  variables  subject  to  not  exceeding 
production  limits,  and  consistency  between  sales,  production,  and  flow  (5.27).  The 
KKT  conditions  for  each  of  the  producers'  problems  are  both  necessary  and  sufficient 
(Bazaraa  et  ah,  1993)  given  the  functions  chosen  and  these  conditions  for  each  of  the 
producers  (producer  A  at  node  1,  producer  B  at  node  1,  producer  C  at  node  2, 
producer  D  at  node  2)  are  as  follows: 


47  All  maximum  values  for  primal  variables  denoted  by  an  overbar  are  assumed  to  be  positive  as  are 
cost  coefficients  yf  ,  yf  ,  y G ,  y% . 
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(5.29) 


(5.30) 


(5.31) 


In  addition  to  the  KKT  conditions  for  the  four  producer  problems,  there  are 
market-clearing  conditions  that  force  supply  to  equal  demand: 


0  = 
0  = 


sf+s?\- 

?2  +  4 


]-  Dx{nx ),  nx  free 
+  /i2  +  fn  ] -  D2  (x2\  n2  free 


(5.32) 


Note  that  the  terms  in  square  brackets  are  the  net  supply  at  each  node 
(assuming  no  losses)  and  D„( jz„),  n  =  1,  2  are  the  nodal  demand  as  a  function  of  the 
price  n„.  While  the  producers  depicted  above  operate  using  the  network,  there  is 
additional  a  transportation  system  operator  (TSO)  who  manages  the  congestion  and 
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flows.  The  TSO's  linear  program  is  as  follows  (where  other  objectives  are  also 
possible): 


max(r/^G  +  r12)g12  -crao(g12) 

812 

S-L  (5.33) 

§12  ^  gl2  (*12) 

&12^0 

Here,  the  TSO  controls  the  variable  g12  which  is  the  flow  from  node  1  to  node 
2,  cTSO(gn )  is  a  network  operations  cost  function  (assumed  linear  i.e., 
c7:so(gl2)  =  yTS°gl2,  yTS°  >  0  )  and  gi2  is  the  capacity  of  the  link  between  nodes  1  and 
2.  The  KKT  conditions  for  this  problem  are  both  necessary  and  sufficient  and  since  it 
is  a  linear  program  and  these  conditions  are  the  following: 

0<-r,2  -Tu  +  r  +£12J-gl2>  0  (5  34) 

0^lTl2-gl2  -L*12  ^0 

The  last  part  of  the  equilibrium  problem  is  the  market-clearing  conditions  that 
balance  the  flow  controlled  by  the  network  operator  and  thus  by  producers  A  and  B: 

0  =  gn-[/i"+/?hl2free  (5.35) 

The  LCP  for  this  energy  network  problem  is  thus  the  KKT  conditions  of  the 
producers:  (5.28),  (5.29),  (5.30),  (5.31),  the  nodal  market-clearing  conditions  (5.32), 
the  KKT  conditions  of  the  TSO  (5.34)  and  the  market-clearing  conditions  of  the 
transportation  market  (5.35).  Figure  5.5  below  shows  a  diagrammatic  representation 
of  this  network. 
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Producer  A  Producer  C 


Figure  5.5:  Representation  of  Second  Network  Example 

In  this  problem,  sf ,  sf ,  s2  ,  s2  ,  qf  ,  qf ,  q2  ,  q2  are  the  variables  that  are 
integer-constrained  in  variations  3  and  4.  The  goal  is  to  find  a  solution  which  has 
these  variables  as  integers.  Note  that  there  are  multiple  integer  solutions.  The  values 
for  the  input  parameters  as  well  as  the  six  variations  that  were  tested  are  shown  in 
Table  5.11  below. 
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Table  5.12:  Description  of  Formulation  Variations  for  Second  Network  Example 


Variation 

^-Complementary 

e-Integrality 

Problem  Description 

1 

No 

No 

MLCP 

3 

No 

No 

Integer  variables 

4 

Yes 

No 

Integer  variables 

5 

Yes 

No 

Continuous  variables 

6 

No 

Yes 

Continuous  variables 

7 

Yes 

Yes 

Continuous  variables 

As  in  the  DC-Nash  example,  several  numerical  variations  were  done  to  see  the 
change  in  solutions.  Variation  1  was  a  mixed-complementary  problem  (MCP)  without 
imposing  integer  restrictions.  Variation  3  involved  converting  the  MCP  to  a 
formulation  with  disjunctive  constraints  but  restricting  the  variables  of  production  and 
sales  to  be  integer.  The  rest  of  the  variations  then  go  through  the  different 
combinations  as  in  the  DC-Nash  example. 

First,  variations  4  and  7  give  an  integer  solution.  However,  due  to  the  presence 
of  multiple  equilibria,  these  solutions  need  not  be  unique  as  in  the  DC-NASH  game. 
Multiple  starting  points  were  chosen,  and,  according  to  the  numerical  tests,  the 
reported  solution  had  the  highest  objective  function  value  (along  with  some  other 
equilibria  not  reported)  when  a  feasible  integer  solution  was  desired.  Hence, 
variations  4  and  7  can  be  used  to  obtain  optimal,  integer  solutions  that  are  feasible. 
Note  that  variation  6  targets  integers  through  s-complementarity,  while  variation  4 
actually  restricts  solutions  to  integer  values. 
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Similar  to  the  previous  example,  variation  1  yielded  a  non-integer  but  optimal 
and  feasible  solution  while  variation  3  was  infeasible.  Again,  this  shows  that  'Q- 
complementarity  is  essential  to  obtain  a  feasible  integer  solution  (as  in  variations  4 
and  7).  However,  only  ^-complementarity  is  not  enough  to  obtain  integer  solutions 
(variation  5)  nor  is  only  s-complementarity  (variation  6). 

The  extra  advantage  of  using  variations  4  and  7  is  that  values  of  dual  variables 
can  be  obtained  and  interpreted.  It  is  interesting  to  note  that  the  dual  variables  change 
from  the  continuous  to  the  integer  case,  which  is  what  was  expected.  However,  it  also 
shows  the  differences  in  solutions  with  relaxation  of  integer  variables  to  solve  a 
problem  and  how  it  leads  to  solutions  that  can  be  very  different  from  the  market 
dynamics  of  an  integer  constrained  problem.  Tables  5.13  and  5.14  below  display  the 
results  obtained  from  this  network  example. 
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Table  5.13:  Results  for  Second  Network  Problem  (Integer  Variables) 


Table  5.14:  Results  for  Second  Network  Problem  (Other  Variables) 


Variations 

1 

3 

4 

5 

6 

7 

fA 

'12 

2.560 

Infeasible 

2.000 

2.000 

2.000 

2.000 

fn 

2.440 

Infeasible 

3.000 

3.000 

3.000 

3.000 

4 

2.000 

Infeasible 

2.000 

2.000 

2.000 

2.000 

4 

0 

Infeasible 

0 

0 

0 

0 

4 

0.250 

Infeasible 

0.500 

0.250 

0.250 

0.500 

4 

0 

Infeasible 

0 

0 

0 

0 

<§12 

5.000 

Infeasible 

5.000 

5.000 

5.000 

5.000 

^12 

2.250 

Infeasible 

2.500 

2.250 

2.250 

2.500 

12.000 

Infeasible 

12.000 

12.000 

12.000 

12.000 

12.000 

Infeasible 

12.000 

12.000 

12.000 

12.000 

8c2 

15.250 

Infeasible 

15.500 

15.250 

15.250 

15.500 

^2 

17.581 

Infeasible 

18.000 

18.000 

18.000 

18.000 

7Ti 

12.000 

Infeasible 

12.000 

12.000 

12.000 

12.000 

n2 

15.250 

Infeasible 

15.500 

15.250 

15.250 

15.500 

>1l2 

2.750 

Infeasible 

3.000 

2.750 

2.750 

3.000 
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5.5.  Summary 


This  chapter  improves  a  methodology  to  solve  discretely-constrained  Nash  games 
formulated  as  mixed  complementarity  problems.  The  discrete  restrictions  can  lead  to 
infeasible  solutions,  so  a  relaxation  is  needed.  Along  with  providing  both  a 
complementary  and  integrality  relaxation,  this  chapter  uses  the  technique  from 
Chapter  4  to  solve  the  resulting  DC-MLCP. 

The  two-level  formulation  proposed  in  this  chapter  gives  a  new  way  to  look  at 
an  otherwise  one-level  problem.  First,  this  fonnulation  shows  that  there  are  actually 
two  different  sets  of  optimization  problems  hidden  in  this  one-level  problem.  One  set 
of  minimization  problems  aims  to  minimize  deviations  from  complementary  and 
integrality.  The  other  set  of  complementary  problems  aims  to  solve  either  a  Nash- 
Cournot  game  or  a  network  problem  (as  the  two  examples  discussed  in  this  chapter; 
there  can  be  other  applications).  In  this  way,  the  two-level  approach  tackles  the 
problem  from  a  different  perspective.  This  perspective,  coupled  with  the  solution 
technique  from  Chapter  4,  ends  up  performing  better  than  the  single-stage  method  of 
(Gabriel  et  ah,  2011a),  (Gabriel  et  ah,  2011b). 

From  the  theoretical  analysis  carried  out  and  the  examples  considered,  several 
conclusions  can  be  drawn.  First,  relaxing  both  integrality  and  complementarity  in  the 
lower-level  problem  while  using  the  upper-level  to  minimize  deviations  enables  the 
selection  of  an  integer,  equilibrium  solution.  Second,  the  method  of  SOS  Type  1 
variables  from  Chapter  4  proves  to  be  computationally  quicker  than  the  method  of 
disjunctive  constraints  for  two  of  the  numerical  examples  illustrated  in  this  chapter. 
Third,  different  variations  of  relaxation,  as  shown  by  Variations  1  through  7  for  each 
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example,  can  lead  to  different  solutions.  It  also  helps  analyze  the  importance  of  the 
technique  to  practical  examples. 
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Chapter  6:  Conclusions 


This  chapter  provides  a  summary  of  the  work  done  in  the  dissertation.  The 
dissertation  went  through  three  different  types  of  two-level  problems,  and  provided 
novel  solution  techniques  for  each  of  them.  Several  applications  of  these  techniques 
were  provided  to  show  the  nuances  of  each  method.  This  chapter  will  start  off  with 
concluding  remarks  about  each  two-level  problem  studied.  Then,  the  main 
contributions  of  this  dissertation  will  be  listed.  Finally,  some  proposals  for  future 
work  as  an  extension  of  the  work  provided  in  this  dissertation  will  be  presented. 

6.1.  Concluding  Remarks 

6.1.1.  Robust  Optimization 

Numerous  robust  optimization  techniques  exist  in  the  literature,  but  the  goal  of  this 
dissertation  was  to  develop  a  technique  which  is  numerically  more  efficient  than 
previous  techniques.  Chapter  3  presented  a  method  based  on  Benders  decomposition, 
which  was  shown  to  be  numerically  more  efficient  when  compared  to  a  previous 
method  (Li  et  al.,  2006). 

The  modified  Benders  method  was  also  shown  to  be  computationally 
tractable,  in  that  empirically  the  increase  in  the  number  of  function  calls  was  at  most 
linear  with  an  increase  in  variables.  Diverse  numerical  examples  were  provided  to 
show  the  applicability  of  the  method  to  different  types  of  problems. 

Previous  methods  exist,  which  could  easily  solve  linear  and  quadratic  robust 
optimization  problems  efficiently.  But  many  engineering  design  problems,  as  also 
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stated  in  this  dissertation,  involve  nonlinear  constraints  and  objective  functions.  The 
modified  Benders  method  was  shown  to  be  able  to  tackle  these  problems,  and  a 
sampling  technique  was  provided  so  the  user  could  choose  the  level  of  accuracy 
desired.  In  particular,  the  engineering  design  examples  showed  how  the  selection  of 
an  optimal  design  can  vary  with  the  presence  of  uncertainty.  Moreover,  the  examples 
also  showed  how  the  presence  of  uncertainty  degrades  objective  function 
perfonnance. 

A  final  example  showed  the  importance  of  studying  uncertainty  to 
environmental  market  strategies.  The  future  of  a  carbon  tax  and  retrofitting 
technology  is  uncertain,  and  this  uncertainty  in  future  events  can  have  important 
implications  on  decisions  made  today.  As  shown  in  the  example,  the  uncertainty  of  a 
tax  discourages  energy  intensive  infrastructure  for  today,  when  the  decision  maker  is 
extremely  risk  averse.  Worries  of  a  larger  tax  in  the  future  encourages  infrastructure 
to  not  be  as  energy  intensive  under  a  worst-case  scenario. 

The  approach  presented  in  this  dissertation  is  designed  for  robust  optimization 
problems  with  a  goal  to  decrease  computational  time.  One  drawback  of  this  approach 
is  that  there  is  no  built-in  verification  that  the  solution  is  actually  robust  for  nonlinear 
robust  optimization  problems.  Other  methods  (Gunawan  &  Azarm,  2004),  (Li  et  ah, 
2006)  have  optimization  problems  that  verify  the  solution  is  robust  within  the 
approach.  Secondly,  this  approach  requires  explicit  objective  and  constraint  functions 
to  be  able  to  work.  Simulation  or  “black  box”  type  problems  will  not  be  solved  using 
the  approach  presented  in  this  dissertation.  Moreover,  this  dissertation  only  provides 
numerical  evidence  that  the  modified  Benders  method  works  for  robust  optimization 
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problems  with  quasiconvex  constraints.  A  mathematical  proof  would  improve  the 
validity  of  the  method  and  is  an  option  for  future  work.  Another  area  of  improvement 
is  a  better  sampling  technique  than  the  one  presented  for  nonlinear  constraint 
functions.  Improvements  in  sampling  and  theory  might  eventually  lead  to  a  solution 
technique  for  general  nonlinear  robust  optimization  problems. 

6.1.2.  MPECs  and  EPECs 

Chapter  4  describes  a  new  solution  technique  for  MPECs  and  EPECs,  which  was 
developed  to  solve  large-scale  problems  such  as  the  North  American  gas  model 
MPEC.  The  new  technique  was  developed  to  be  more  computationally  efficient  than 
previous  techniques  for  solving  these  problems.  A  lot  of  times  optimization  solvers 
return  a  solution  as  infeasible  if  they  are  unable  to  find  one  for  complex  problems. 
The  aim  was  to  develop  a  simple  enough  technique  that  could  be  applied  to  a  wide 
variety  of  problems. 

The  two  algorithms  presented  in  Chapter  4  were  shown  to  solve  large  MPECs 
with  much  less  computational  effort  when  compared  to  disjunctive  constraints  and 
also  be  applicable  to  complex  problems  (e.g.,  the  North  American  gas  model)  where 
traditional  methods  had  failed.  The  methods  were  restricted  to  be  used  in  problems 
where  the  KKT  conditions  were  necessary  and  sufficient,  which  decreases  the 
applicability  of  various  functional  forms  but  still  lends  itself  to  different  examples. 

The  focus  of  the  examples  was  on  different  types  of  Stackelberg  games  and 
Nash-Cournot  games  with  a  modified  structure.  The  theme  of  the  examples  was 
energy  and  natural  gas  production,  outlined  by  the  North  American  gas  model. 
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Various  scenarios  for  shale  gas  in  the  United  States  were  developed  using  the 
North  American  gas  model.  The  scenarios  studied  what  would  happen  under  a  tax  for 
shale  production,  a  tax  on  all  natural  gas  production,  and  the  presence  of  more  shale 
than  predicted.  One  of  the  main  conclusions  was  that  in  the  presence  of  a  tax,  the 
producers  pass  the  tax  onto  the  consumers.  Moreover,  the  top-level  firm  always 
makes  the  majority  of  the  profits  by  manipulating  the  market.  The  reality  of  the 
market  situation  probably  lies  somewhere  in  between  that  of  a  Stackelberg  game  and 
a  Nash-Coumot  game,  but  being  able  to  study  this  fonnulation  was  instructive. 

6.1.3.  Discretely-Constrained  Mixed  Linear  Complementarity  Problems 

The  technique  presented  in  Chapter  5  was  a  new  way  to  solve  and  think  about 
discretely-constrained  mixed  linear  complementary  problems.  The  technique 
provided  a  way  to  solve  a  relaxed  version  of  the  problem  in  one  stage,  thus  converting 
a  two-level  problem  into  one  level.  This  conversion  into  one  stage  was  initially 
achieved  using  disjunctive  constraints,  but  this  meant  the  solution  would  depend  on  a 
large  constant.  This  dissertation  used  the  techniques  presented  in  Chapter  4  to  not 
have  to  use  disjunctive  constraints  when  finding  solutions  to  DC-MLCPs. 

The  first  set  of  numerical  examples  studied  discretely-constrained  Nash- 
Cournot  games.  While  the  continuous  versions  of  these  games  have  been  extensively 
studied,  imposing  discrete  restrictions  might  lead  to  infeasibility.  Hence,  relaxing  the 
integer  restrictions  as  well  as  the  complementary  conditions,  while  targeting  specific 
integer  values,  provided  a  solution  to  these  games.  The  computational  effort  was  also 
abated  using  SOS  Type  1  variables  as  opposed  to  disjunctive  constraints.  In  all 
examples,  the  payoffs  for  the  players  in  the  continuous  relaxation  were  higher  than 
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with  the  integer  restrictions.  Two  different  variations  were  provided  to  be  able  to 
obtain  integer  solutions  to  DC-Nash  games. 

The  second  set  of  numerical  examples  studied  discretely-constrained  network 
problems.  Network  problems  can  also  be  expressed  as  complementary  problems,  and 
adding  the  discrete  restrictions  would  yield  the  same  problems  of  infeasibility  as  the 
discretely-constrained  Nash  games.  Again,  different  variations  were  studied  to  see 
which  one  yielded  an  integer  solution.  Two  variations,  the  same  ones  that  worked  for 
the  discretely-constrained  Nash  games,  worked  for  these  network  problems  as  well. 
Dual  variables  are  often  used  in  network  problems  to  obtain  shadow  prices,  and  this 
technique  helped  obtaining  these  prices.  However,  the  applicability  of  these  prices  is 
still  a  matter  of  debate  as  the  complementary  and  integrality  relaxations  also  factor 
into  these  prices. 

Another  advantage  of  using  the  technique  outlined  in  this  dissertation  was  that 
a  tradeoff  between  complementary  and  integrality  can  be  obtained.  This  was 
numerically  shown  by  studying  different  variations  in  Chapter  5. 

6.2.  Main  Contributions 

This  dissertation  is  focused  on  solving  three  specific  types  of  two-level  problems. 
However,  these  three  types  of  problems  have  been  chosen  to  be  the  ones  that  best 
encompass  the  class  of  two-level  problems.  First,  robust  optimization  is  a  two-level 
problem  where  the  lower-level  can  be  thought  of  as  checking  the  feasibility  of  an 
upper-level  decision.  In  this  way,  the  lower-level  aims  to  check  feasibility,  but  does 
not  have  an  objective  or  goal  for  itself.  For  MPECs  and  EPECs,  the  lower-level  is 
either  a  cooperative  or  noncooperative  equilibrium  may  or  may  not  conflict  with  the 
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upper-level  objective.  While  the  lower-level  alters  the  feasible  space  for  the  upper- 
level  problem,  the  focus  is  on  influencing  the  objective  function  of  the  upper-level 
problem.  Thus,  these  two  types  of  two-level  problems  encompass  dealing  with 
influence  of  the  lower-level  on  constraints  (robust  optimization)  and  objective 
function  (MPECs)  of  the  upper-level  directly,  and  indirectly  the  objective  function 
(robust  optimization)  and  constraints  (MPECs)  of  the  upper-level.  Finally,  the  third 
type  of  problem  is  something  which  starts  off  with  a  one-level  structure,  but  is 
converted  to  two  levels  to  be  able  to  solve  more  easily. 

The  first  main  contribution  of  this  dissertation  is  the  application  of 
decomposition  techniques  to  two-level  problems,  which  helps  convert  them  to  a 
single  one-level  problem  (as  in  the  case  of  MPECs,  EPECs,  and  DC-MLCPs),  or  a 
series  of  one-level  problems  that  can  be  solved  iteratively  (as  in  the  case  of  robust 
optimization).  This  use  of  decomposition  techniques  provides  insight  that  could  not 
be  achieved  through  a  two-level  analysis,  for  example,  the  robust  feasible  region  for 
robust  optimization  problems,  the  absolute  value  function  equality  in  MPECs  and 
EPECs,  and  obtaining  shadow  prices  from  DC-MLCPs.  These  decomposition 
techniques  are  presented  in  a  way  to  take  advantage  of  the  problem  structure,  and 
obtain  a  solution  that  can  relate  to  the  original  problem. 

The  second  main  contribution  of  this  dissertation  is  to  provide  methods  that 
greatly  speed  up  computation  time  for  two-level  problems.  Robust  optimization 
problems  have  been  traditionally  solved  using  a  nested  inner-outer  structure  which 
takes  a  lot  of  computational  effort.  MPECs  and  EPECs  have  been  solved  using 
primarily  disjunctive  constraints  which  not  only  involve  great  computational  effort 
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because  of  the  presence  of  binary  variables,  but  also  require  the  selection  of  a  large 
constant  which  is  not  immediately  obvious.  DC-MLCPs  have  been  solved  either 
successively  fixing  and  relaxing  discrete  variables  or  using  disjunctive  constraints, 
both  of  which  are  computationally  more  expensive  than  the  methods  provided  in  this 
dissertation.  In  particular,  the  method  for  MPECs  can  be  applied  to  a  host  of  other 
problems  to  speed  up  computation  wherever  a  product  of  two  terms  resulting  in  a 
nonlinear  function  is  present. 

The  third  main  contribution  of  this  dissertation  is  applying  the  theory  to  an 
extremely  diverse  set  of  examples.  The  dissertation  contains  examples  from 
environmental  markets,  energy  markets,  power  systems,  structural  optimization, 
engineering  design,  networks,  and  game  theory.  These  same  examples  can  also  be 
split  into  academic  subjects  of  operations  research,  economics,  mechanical 
engineering,  and  market  design.  A  host  of  such  examples  serves  the  academic 
community  well,  as  it  outlines  the  importance  of  research  into  the  theory  of  two-level 
problems. 

6.3.  Future  Research 

6.3.1,  Multiobjective  Mixed-Integer  Robust  Optimization 

There  are  two  natural  ways  to  develop  the  ideas  presented  for  robust  optimization. 
These  ideas  arise  out  of  the  methods  developed  in  the  dissertation,  and  it  is 
convenient  that  this  direction  is  shared  by  current  research  as  well. 

Many  engineering  design  applications  involve  multiobjective  optimization. 
Thus,  extending  the  modified  Benders  method  to  be  applicable  to  multiobjective 
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robust  optimization  problems  would  be  useful.  Since  the  modified  Benders  method 
has  already  converted  the  two-level  problem  into  a  single-level,  combining  it  with 
traditional  methods  of  multiobjective  optimization  would  be  natural.  For  example,  if 
the  robust  feasible  region  is  provided,  any  multiobjective  method  can  be  applied. 
Hence,  each  stage  of  the  modified  Benders  decomposition  method  can  involve 
solving  a  multiobjective  problem.  Since  the  modified  Benders  method  is  gradient- 
based,  it  would  make  sense  to  combine  it  with  another  gradient-based  method  such  as 
Normal  Boundary  intersection  or  one  of  its  variations  (Siddiqui  et  ah,  201  Id).  Each 
step  of  the  Normal  Boundary  intersection  method  provides  one  point  on  the  Pareto 
frontier.  The  modified  Benders  method  would  be  used  at  each  step  to  come  up  with 
one  robust  Pareto  point,  thus  generating  a  robust  Pareto  frontier. 

The  second  natural  extension  has  to  do  with  solving  mixed-integer  robust 
optimization  problems.  Standard  Benders  decomposition  is  already  applicable  to 
mixed-integer  optimization  problems.  Thus,  a  variation  can  easily  be  considered 
which  contains  standard  Benders  cuts  and  modified  Benders  cuts  to  solve  a  robust 
mixed-integer  optimization  problem.  These  two  ideas  can  then  be  combined  to  solve 
a  mixed-integer  robust  optimization  problems. 

6.3.2,  Solving  Nonlinear  MPECs  and  EPECs 

The  methods  presented  in  this  dissertation  were  only  applicable  to  MPECs  and 
EPECs  which  comprised  of  optimization  problems  where  the  KKT  conditions  were 
necessary  and  sufficient.  However,  to  obtain  local  solutions  to  nonlinear  programs  an 
approximation  scheme  can  be  developed  where  the  lower  level  problem  is  locally 
approximated.  This  can  be  done  using  SOS  Type  2  variables,  and  the  linear 


188 


interpolation  could  have  the  KKT  conditions  necessary  and  sufficient  within  a 
specified  interval. 

An  easier  task  would  be  to  consider  the  case  where  the  KKT  conditions  might 
just  be  necessary  (or  sufficient)  and  develop  an  approximation  scheme  from  there.  In 
particular,  if  the  product  of  two  terms  becomes  complicated,  other  approximation 
techniques  may  be  studied. 

6.3.3.  Solving  Large-Scale  Mixed-Integer  Complementary  Problems 

The  relaxation  techniques  put  into  the  DC-MLCPs  in  Chapter  5  were  not  put  to  the 
test  on  larger  problems.  There  might  be  even  better  ways  to  approximate  the 
relaxation  of  complementarity.  For  example,  a  nonlinear  function  describing  the 
product  might  be  added  as  a  constraint. 

In  many  cases,  the  formulation  might  yield  a  simple  way  to  both  add 
relaxations  and  approximate  the  lower-level  product  at  the  same  time.  This  can  then 
be  tested  on  large  mixed-integer  complementarity  problems,  solutions  to  which  can 
be  very  useful  when  studying  market  or  network  dynamics  problems. 
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Appendices 


Appendix  A:  Robust  Optimization  Test  Problems 

For  examples  2  to  4:  qx  =-l ,q2  =  -l,A, qv  =  0.1,  A qT  =  0.1,Ax3.  =  0.1 . 

(Example  2) 

min|(x,  -0.6)"  +  (x2  -0.6)"  -x3  -  x4  +loj 
s.t. 

(ql+ql)  +  xl+x2<0 

(q2+q2)  +  x2+x4  <0 

-Xj  <  0,  -  x2  <  0,  -  x3  <  0,  -  x4  <  0 

where :  \/qx  {-Aql,Aql} ,\/q2  &[-Aq2,Aq2j 


Table  Al:  Solution  to  Example  2 


Information 

Nominal 

Solution 

Robust 

Solution 

Li  et  al."s  (2006) 

Solution 

Xi 

0.5 

0.45 

0.375 

x2 

0.5 

0.45 

0.375 

As 

1 

0.90 

0.416 

X4 

0 

0 

0.416 

Ax) 

9.02 

9.145 

9.268 

Function  Calls 

7 

19 

2592 

(Al) 
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(Example  3) 

min^Xj  -0.6)"  +  (x2  -0.6)"  -x3  -x4  +  loj 
s.t. 

( ql  +qx)  +  X\+x2  <0 

(q2  +  <72)  +  x3  +x4  <  0 

— Xj  <  0,  -  x2  <  0,  -  x3  <  0,  -  x4  <  0 

where :  g  { — At/, ,  A qx } ,  \/q2  e  {-A q2 ,  A q2 } ,  Vx3  e  |x3  -  Ax3 ,  x3  +  Ax3 }  (A2) 


Table  A2:  Solution  to  Example  3 


Information 

Nominal 

Solution 

Robust 

Solution 

Li  et  al."s  (2006) 

Solution 

X; 

0.5 

0.45 

0.375 

X2 

0.5 

0.45 

0.375 

X3 

1 

0.10 

0.416 

X4 

0 

0.70 

0.416 

Ax) 

9.02 

9.145 

9.268 

Function  Calls 

7 

21 

2808 

(Example  4) 

min|(xj  -0.6)"  +(x2  -0.6)"  -x3x4  +loj 
s.t. 

(ql+ql)  +  xl+x2<0 

(q2  +^2)  +  x3  +x4  <  0 

-Xj  <  0,-x2  <  0,-x3  <  0,-x4  <  0 

where  \/qx  e  {-A q^ ,  A qx } ,  \/q2  e  {-A^2 ,  A q2 } ,  Vx3  e  |x3  -  Ax3 ,  x3  +  Ax3 } 
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Table  A3:  Solution  to  Example  4 


Information 

Nominal 

Solution 

Robust 

Solution 

Li  et  al."s  (2006) 

Solution 

X; 

0.5 

0.45 

0.40 

X2 

0.5 

0.45 

0.40 

X3 

1 

0.90 

0.40 

X4 

0 

0 

0.40 

fix) 

9.77 

9.8850 

9.92 

Function  Calls 

7 

21 

2592 

(Example  5) 

Ax  =  Ap  =  0. 1 

min(2x1  +  3x2  -  5x3  -  2x4  +  3x5 ) 

X 

S.t. 

xj  +  x2  -  2x3  -  x4  +  3x5  - 1  >  0 
2xj  -  2x2  +  3x3  -  x4  +  x5  -  3  >  0 

-  5  <  x;  <  5,  i  =  1,2 

-  5  <  x;  <  5,  i  =  3,4,5 


(A4) 
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Table  A4:  Solution  to  Example  5 


Information 

Nominal 

Solution 

Robust 

Solution 

Li  et  al."s  (2006) 

Solution 

X; 

-4.00 

-3.40 

1.04 

*2 

-5.00 

-4.90 

-4.53 

.*4 

5.00 

5.00 

5.00 

X4 

-5.00 

-5.00 

-0.37 

5.00 

5.00 

5.00 

Ax) 

-23.00 

-21.50 

-20.75 

Function  Calls 

5 

17 

7856 

(Example  6) 

At.  =  Ap  =  0. 1 

min(2.1xj  +  3.07x,  -5x3  -2x4  +2.4x5) 

X 

S.t. 

0.9xj  +x2  -  2.2x3  -1.1x4  +3.5x5  -1.2  >  0 
2xj  - 2x2  +  3x3  -  x4  +  x5  - 10  >  0 

-  5  <  x;  <  5,  i  =  1,2 

-  5  <  x;  <  5,  i  =  3,4,5 


(A5) 
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Table  A5:  Solution  to  Example  6 


Information 

Nominal 

Solution 

Robust 

Solution 

Li  et  al."s  (2006) 

Solution 

X; 

-5.00 

-4.90 

-4.23 

X2 

-5.00 

-4.90 

-4.52 

Xs 

5.00 

5.00 

5.00 

X4 

-3.82 

-4.27 

-3.67 

x5 

5.00 

5.00 

5.00 

Ax) 

-31.21 

-29.79 

-28.36 

Function  Calls 

5 

17 

11099 

(Hock  100) 

This  is  problem  100  modified  from  (Hock  &  Schittkowski,  1980).  At.  =  Ap  =  0.1 
minUXj  -10)2  +  5(x2  -12)"  +  x34  +  3(x4  —  1 1)2  +  10x56  +  7x62  +  x74  -4x6x7  -  10x6  -8x7 

X  ' 

S.t. 

127-2xfi  -3x24  -x3  -4x4“  -5x5  >  0 
282  -  7xj  -  3x2  -  10x32  -  x4  +  x5  >0 
196-23xj  -x22  -6x62  +8x7  >  0 
-  4xj2  -  x22  +  3xjX2  -  2x3"  -  5x6  + 1  1x7  >0 

(A6) 
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Table  A6:  Solution  to  Hock  100 

Information 

Nominal 

Solution 

Robust 

Solution 

Li  et  al."s  (2006) 

Solution 

Xl 

2.3304 

2.2350 

- 

X2 

1.9514 

1.8546 

X3 

-0.4775 

-0.4749 

X4 

4.3657 

4.3533 

X5 

-0.6245 

-0.6251 

X6 

1.0381 

1.0359 

x 7 

1.5942 

1.5970 

A*) 

680.6301 

692.3847 

Function  Calls 

7 

19 

>To9 

Bj 


(Hock  106) 

At.  =  Ap  =  0. 1 


min  ((jCj )  +  (x,  )  +  (x3 )) 

X 

S.t. 

1  -  0.0025(x4  +  x6 )  >  0 
1  -  0.0025(xs  +  x7  -  x4)  >  0 

1  -  0.01(x8  -x5)  >  0  (A7) 

XjX6  -833.33252t4  -IOOx,  +83333.333  >  0 
x2x7  -  1250x5  -  x2x4  +  1250t4  >  0 
x3t8  - 1250000  -  xix5  +  2500t5  >  0 

100  <  Xj  <  10000,1000  <  x2  <  10000,1000  <  x3  <  10000,10  <  X,.  <  1000,  i  =  4,..., 8 
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Table  A7:  Solution  to  Hock  106 

Information 

Nominal 

Solution 

Robust 

Solution 

Li  et  al."s  (2006) 

Solution 

Xl 

579.32 

388.73 

X2 

1359.94 

1540.21 

X3 

5110.07 

5290.11 

X4 

182.02 

150.89 

X5 

295.60 

288.40 

X6 

217.98 

209.11 

x7 

286.42 

262.49 

X8 

395.60 

388.40 

J(x) 

7049.33 

7219.06 

Function  Calls 

5 

17 

MO9 

Bj 


Appendix  B:  Discussion  on  Function  Calls 

One  of  the  benchmarks  of  a  useful  algorithm  is  that  it  is  uses  less  computational  effort 
than  other  algorithms.  One  way  to  measure  computational  effort  of  an  algorithm  is  a 
comparison  of  CPU  time,  i.e.,  how  fast  the  algorithm  can  solve  certain  test  problems 
when  compared  to  others.  However,  CPU  time  can  vary  with  the  type  of  computer 
used,  other  programs  running  in  the  background,  and  other  factors  that  are  machine 
dependent. 

Measuring  the  number  of  function  calls  is  a  measure  of  computational  effort 
that  is  machine  independent.  Moreover,  measuring  computational  efficiency  in  terms 
of  function  calls  can  better  estimate  how  the  algorithm  will  perform  for  different 
types  of  problems  (e.g.,  black  box  or  simulation-based  design). 

This  dissertation  defines  function  calls  as  any  instances  where  the  solver  calls 
an  objective  function,  constraint,  or  other  value  or  assignment  in  the  optimization 
problem.  This  is  based  on  the  definition  of  a  statement  execution  in  GAMS,  which  is 
defined  as  any  instance  where  the  solver  calls  an  equation  or  other  value  or 
assignment  in  the  optimization  problem  (GAMS,  2010).  This  definition  was  chosen  in 
part  because  the  modified  Benders  method  was  programmed  and  tested  in  GAMS. 

This  definition  is  also  similar  to  the  other  definitions  of  function  calls  in  the 
recent  literature.  The  definition  by  (Hu  et  ah,  2011)  is  that  a  “function  call  refers  to 
one  instance  of  calculating  objective  and  constraint  functions  altogether  (i.e.,  one  call 
to  the  optimization  problem).”  The  authors  (Hu  et  al.,  2011)  have  used  MATLAB  to 
solve  their  test  problems,  and  their  definition  depends  on  their  use  of  MATLAB 
(MATLAB,  2008).  MATLAB  does  not  have  an  internal  explicit  function  call  counter 
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like  GAMS,  but  the  method  of  (Hu  et  al,  2011)  entails  putting  the  objective  and 
constraint  functions  in  one  “m-file”  and  attaching  a  function  call  counter  within  this 
file.  Note  it  is  not  possible  to  place  a  similar  counter  in  GAMS  because  the  GAMS 
file  structure  is  different  than  MATLAB.  Another  definition  is  offered  by  (Li  et  ah, 
2011),  who  define  function  calls  “equal  to  the  number  of  points  that  have  been 
evaluated  during  one  run  of  the  optimizer.”  The  authors  in  (Li  et  al.,  2011)  used  the 
solver  XPRESS  (XPRESS,  2003)  for  their  test  problems. 

Since  all  problems  except  one  (Heat  Exchanger  Design  in  Section  3.5.3  was 
solved  using  MATLAB)  were  solved  in  GAMS,  the  following  example  provides  a 
basis  for  comparison  for  function  call  counting  in  GAMS  and  MATLAB.  This  is  the 
nominal  version  of  Fleury"s  weight  minimization  like  problem  (Section  3.5.1). 

N 

min  f(x)  =  YJxi 

x  ~ i 

s.t. 


0.95  N 

z 


1=1 

0.95  N 


z 


;= l 


1  1 

- 1 - 7 

xt  N 

l _ 1_ 

N2 


Z 

i'=0.9  5  Af+1 
N 

z 

i=0.95JV+l 


X,. 

j_ 

Xi 


N<  0 

0.9A  <  0 


<  x,.  <  N2 


i  =  1,2,...,  A 


(Bl) 


This  problem  (Bl)  was  solved  using  both  GAMS  and  MATLAB.  The  values 
of  N  were  changed  to  give  an  idea  of  computational  effort  for  MATLAB  and  GAMS 
(and  two  different  ways  of  measuring  function  calls).  Table  Bl  shows  these  results. 
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Table  Bl:  Comparing  Function  Calls  Between  GAMS  and  MATLAB 


Number  of 

Variables  (AO 

Number  of 

Constraints 

GAMS  Function 

Calls 

MATLAB 

Function  Calls 

100 

102 

506 

5493 

200 

202 

607 

8210 

300 

302 

913 

12252 

400 

402 

1201 

17786 

500 

502 

1403 

20096 

Clearly,  even  though  we  are  careful  in  using  the  same  definition  for 
MATLAB  and  GAMS,  there  is  a  difference  in  counting  function  calls  for  these 
programs.  Roughly,  the  function  calls  in  GAMS  are  an  order  of  magnitude  or  two 
lower  for  example  (Bl).  This  difference  should  be  kept  in  mind  when  looking  at  the 
examples  in  this  dissertation.  Just  to  note,  the  function  calls  for  the  Heat  Exchanger 
example  (Section  3.5.3)  were  reported  using  the  counting  method  for  MATLAB.  The 
results  on  maximum  function  calls  presented  in  Table  3.1  are  also  applicable  to  both 
methods  of  counting  since  they  talk  about  the  maximum  possible  function  calls,  and 
are  based  on  the  definition  of  function  call  in  this  dissertation. 
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